From auton.sysnotify at gmail.com Fri Nov 2 18:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 02 Nov 2018 18:30:06 -0400 Subject: A desktop is down Message-ID: <20181102223006.A98C740F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 2 19:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 02 Nov 2018 19:00:06 -0400 Subject: A desktop is down Message-ID: <20181102230006.A79DD40F8987@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 2 19:30:11 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 02 Nov 2018 19:30:11 -0400 Subject: A desktop is down Message-ID: <20181102233014.580D440F8989@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 2 19:47:55 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Fri, 02 Nov 2018 19:47:55 -0400 Subject: Cron /usr/local/bin/rsnapshot -c /usr/local/etc/rsnapshot_desktops.conf daily Message-ID: <5bdce22b.3499.722bdc98@uranus.int.autonlab.org> ---------------------------------------------------------------------------- rsnapshot encountered an error! The program was invoked with these options: /usr/local/bin/rsnapshot -c /usr/local/etc/rsnapshot_desktops.conf daily ---------------------------------------------------------------------------- ERROR: /usr/bin/rsync returned 255 while processing backup at luna.auton.cs.cmu.edu:/home/xinyul2/ From auton.sysnotify at gmail.com Fri Nov 2 20:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 02 Nov 2018 20:00:06 -0400 Subject: A desktop is down Message-ID: <20181103000007.E0BDD40F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 2 20:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 02 Nov 2018 20:30:06 -0400 Subject: A desktop is down Message-ID: <20181103003006.4A5A040F8987@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 2 21:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 02 Nov 2018 21:00:06 -0400 Subject: A desktop is down Message-ID: <20181103010007.3968440F8987@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 2 21:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 02 Nov 2018 21:30:06 -0400 Subject: A desktop is down Message-ID: <20181103013006.6848940F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 2 22:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 02 Nov 2018 22:00:05 -0400 Subject: A desktop is down Message-ID: <20181103020006.5191840F898B@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 2 22:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 02 Nov 2018 22:30:06 -0400 Subject: A desktop is down Message-ID: <20181103023006.EEA6F40F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 2 23:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 02 Nov 2018 23:00:06 -0400 Subject: A desktop is down Message-ID: <20181103030006.5DA0F40F8989@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 2 23:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 02 Nov 2018 23:30:06 -0400 Subject: A desktop is down Message-ID: <20181103033007.31EBD40F898B@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 00:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 00:00:06 -0400 Subject: A desktop is down Message-ID: <20181103040006.5464140F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 00:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 00:30:06 -0400 Subject: A desktop is down Message-ID: <20181103043006.D055640F8989@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 01:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 01:00:06 -0400 Subject: A desktop is down Message-ID: <20181103050006.5E77B40F8987@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 01:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 01:30:06 -0400 Subject: A desktop is down Message-ID: <20181103053006.980BC40F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 02:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 02:00:05 -0400 Subject: A desktop is down Message-ID: <20181103060006.4513C40F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 02:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 02:30:06 -0400 Subject: A desktop is down Message-ID: <20181103063006.6156D40F8987@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 03:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 03:00:06 -0400 Subject: A desktop is down Message-ID: <20181103070007.6134E40F8989@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 03:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 03:30:06 -0400 Subject: A desktop is down Message-ID: <20181103073007.0D3B840F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 04:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 04:00:06 -0400 Subject: A desktop is down Message-ID: <20181103080006.9573A40F898D@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 04:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 04:30:06 -0400 Subject: A desktop is down Message-ID: <20181103083006.8111040F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 05:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 05:00:05 -0400 Subject: A desktop is down Message-ID: <20181103090006.4893440F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 05:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 05:30:06 -0400 Subject: A desktop is down Message-ID: <20181103093006.CBBAB40F898E@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 06:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 06:00:06 -0400 Subject: A desktop is down Message-ID: <20181103100006.B8AA840F898E@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 06:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 06:30:05 -0400 Subject: A desktop is down Message-ID: <20181103103005.DD18D40F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 07:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 07:00:06 -0400 Subject: A desktop is down Message-ID: <20181103110006.BCB3940F8991@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 07:03:17 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 03 Nov 2018 07:03:17 -0400 Subject: Cron /usr/local/sbin/iocell send -r -u root -h uranus zroot Message-ID: <5bdd8075.38a6.4a253542@bhyve.int.autonlab.org> INFO: sending zroot/iocell to zroot/iocell: /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/done! From auton.sysnotify at gmail.com Sat Nov 3 07:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 07:30:06 -0400 Subject: A desktop is down Message-ID: <20181103113006.E099840F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 08:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 08:00:06 -0400 Subject: A desktop is down Message-ID: <20181103120006.8615740F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 08:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 08:30:06 -0400 Subject: A desktop is down Message-ID: <20181103123007.0C1AD40F898E@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 09:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 09:00:06 -0400 Subject: A desktop is down Message-ID: <20181103130006.CDEB740F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 09:12:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Sat, 03 Nov 2018 09:12:25 -0400 Subject: SMART error (SelfTest) detected on host: uranus Message-ID: <5bdd9eb9.34fd.65ea8658@uranus.int.autonlab.org> This message was generated by the smartd daemon running on: host name: uranus DNS domain: int.autonlab.org The following warning/error was logged by the smartd daemon: Device: /dev/da12 [SAT], Self-Test Log error count increased from 0 to 1 Device info: ST4000NM0024-1HT178, S/N:Z4F03FQ7, WWN:5-000c50-07b0cff9c, FW:SN02, 4.00 TB For details see host's SYSLOG. You can also use the smartctl utility for further investigation. No additional messages about this problem will be sent. From auton.sysnotify at gmail.com Sat Nov 3 09:12:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Sat, 03 Nov 2018 09:12:25 -0400 Subject: SMART error (OfflineUncorrectableSector) detected on host: uranus Message-ID: <5bdd9eb9.34fb.46031d83@uranus.int.autonlab.org> This message was generated by the smartd daemon running on: host name: uranus DNS domain: int.autonlab.org The following warning/error was logged by the smartd daemon: Device: /dev/da12 [SAT], 8 Offline uncorrectable sectors Device info: ST4000NM0024-1HT178, S/N:Z4F03FQ7, WWN:5-000c50-07b0cff9c, FW:SN02, 4.00 TB For details see host's SYSLOG. You can also use the smartctl utility for further investigation. No additional messages about this problem will be sent. From auton.sysnotify at gmail.com Sat Nov 3 09:12:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Sat, 03 Nov 2018 09:12:25 -0400 Subject: SMART error (CurrentPendingSector) detected on host: uranus Message-ID: <5bdd9eb9.34f9.344788dd@uranus.int.autonlab.org> This message was generated by the smartd daemon running on: host name: uranus DNS domain: int.autonlab.org The following warning/error was logged by the smartd daemon: Device: /dev/da12 [SAT], 8 Currently unreadable (pending) sectors Device info: ST4000NM0024-1HT178, S/N:Z4F03FQ7, WWN:5-000c50-07b0cff9c, FW:SN02, 4.00 TB For details see host's SYSLOG. You can also use the smartctl utility for further investigation. No additional messages about this problem will be sent. From auton.sysnotify at gmail.com Sat Nov 3 09:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 09:30:06 -0400 Subject: A desktop is down Message-ID: <20181103133007.281AE40F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 10:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 10:00:06 -0400 Subject: A desktop is down Message-ID: <20181103140006.EA61840F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 10:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 10:30:05 -0400 Subject: A desktop is down Message-ID: <20181103143006.11EF740F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 11:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 11:00:06 -0400 Subject: A desktop is down Message-ID: <20181103150006.C7E7940F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 11:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 11:30:06 -0400 Subject: A desktop is down Message-ID: <20181103153006.720AB40F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 12:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 12:00:05 -0400 Subject: A desktop is down Message-ID: <20181103160006.321CB40F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 12:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 12:30:06 -0400 Subject: A desktop is down Message-ID: <20181103163006.C61F740F898F@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 13:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 13:00:06 -0400 Subject: A desktop is down Message-ID: <20181103170006.4D07F40F898F@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 13:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 13:30:06 -0400 Subject: A desktop is down Message-ID: <20181103173006.4821340F898E@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 14:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 14:00:06 -0400 Subject: A desktop is down Message-ID: <20181103180006.D442640F898E@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 14:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 14:30:06 -0400 Subject: A desktop is down Message-ID: <20181103183006.8023140F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 15:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 15:00:05 -0400 Subject: A desktop is down Message-ID: <20181103190006.3CC9740F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 15:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 15:30:06 -0400 Subject: A desktop is down Message-ID: <20181103193006.B2A1340F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 16:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 16:00:05 -0400 Subject: A desktop is down Message-ID: <20181103200006.2CDCB40F898F@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 16:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 16:30:06 -0400 Subject: A desktop is down Message-ID: <20181103203007.5358C40F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 17:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 17:00:05 -0400 Subject: A desktop is down Message-ID: <20181103210006.2C7EE40F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 15:13:05 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 15:13:05 -0400 (EDT) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181103212056.6A1B21596268@gpu2.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Traceback (most recent call last): File "/usr/sbin/yum-cron", line 729, in main() File "/usr/sbin/yum-cron", line 726, in main base.updatesCheck() File "/usr/sbin/yum-cron", line 618, in updatesCheck self.populateUpdateMetadata() File "/usr/sbin/yum-cron", line 422, in populateUpdateMetadata self.pkgSack # honor skip_if_unavailable File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 1074, in pkgSack = property(fget=lambda self: self._getSacks(), File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 778, in _getSacks self.repos.populateSack(which=repos) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 347, in populateSack self.doSetup() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1477, in _commonLoadRepoXML if self._latestRepoXML(local): File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1446, in _latestRepoXML oxml = self._saveOldRepoXML(local) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1303, in _saveOldRepoXML shutil.copy2(local, old_local) File "/usr/lib64/python2.7/shutil.py", line 130, in copy2 copyfile(src, dst) File "/usr/lib64/python2.7/shutil.py", line 83, in copyfile with open(dst, 'wb') as fdst: IOError: [Errno 28] No space left on device: '/var/cache/yum/x86_64/7/SCL-core/repomd.xml.old.tmp' /etc/cron.hourly/ghc-doc-index: /usr/bin/ghc-doc-index: line 27: /var/lib/ghc/pkg-dir.cache.new: No space left on device diff: /var/lib/ghc/pkg-dir.cache.new: No such file or directory haddock: internal error: .: copyFile: resource exhausted (No space left on device) From auton.sysnotify at gmail.com Sat Nov 3 17:20:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 17:20:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212101.923241596240@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 17:06:38 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 17:06:38 -0400 (EDT) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181103212101.0AB581596240@gpu2.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Traceback (most recent call last): File "/usr/sbin/yum-cron", line 729, in main() File "/usr/sbin/yum-cron", line 726, in main base.updatesCheck() File "/usr/sbin/yum-cron", line 618, in updatesCheck self.populateUpdateMetadata() File "/usr/sbin/yum-cron", line 422, in populateUpdateMetadata self.pkgSack # honor skip_if_unavailable File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 1074, in pkgSack = property(fget=lambda self: self._getSacks(), File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 778, in _getSacks self.repos.populateSack(which=repos) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 347, in populateSack self.doSetup() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1477, in _commonLoadRepoXML if self._latestRepoXML(local): File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1446, in _latestRepoXML oxml = self._saveOldRepoXML(local) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1303, in _saveOldRepoXML shutil.copy2(local, old_local) File "/usr/lib64/python2.7/shutil.py", line 130, in copy2 copyfile(src, dst) File "/usr/lib64/python2.7/shutil.py", line 83, in copyfile with open(dst, 'wb') as fdst: IOError: [Errno 28] No space left on device: '/var/cache/yum/x86_64/7/SCL-core/repomd.xml.old.tmp' /etc/cron.hourly/ghc-doc-index: /usr/bin/ghc-doc-index: line 27: /var/lib/ghc/pkg-dir.cache.new: No space left on device diff: /var/lib/ghc/pkg-dir.cache.new: No such file or directory haddock: internal error: .: copyFile: resource exhausted (No space left on device) From auton.sysnotify at gmail.com Sat Nov 3 17:10:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 17:10:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212101.977971596243@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 17:00:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 17:00:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212103.2299F1596240@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 16:50:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 16:50:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212103.69DBF1596241@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 15:10:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 15:10:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212106.5D82C1596242@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 16:40:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 16:40:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212104.26C971596240@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 16:10:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 16:10:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212105.763511596244@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 16:30:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 16:30:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212104.BC6D41596240@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 16:20:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 16:20:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212105.2FCB41596241@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 15:30:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 15:30:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212105.8B21B1596246@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 16:10:45 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 16:10:45 -0400 (EDT) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181103212106.8CFB71596243@gpu2.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Traceback (most recent call last): File "/usr/sbin/yum-cron", line 729, in main() File "/usr/sbin/yum-cron", line 726, in main base.updatesCheck() File "/usr/sbin/yum-cron", line 618, in updatesCheck self.populateUpdateMetadata() File "/usr/sbin/yum-cron", line 422, in populateUpdateMetadata self.pkgSack # honor skip_if_unavailable File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 1074, in pkgSack = property(fget=lambda self: self._getSacks(), File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 778, in _getSacks self.repos.populateSack(which=repos) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 347, in populateSack self.doSetup() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157, in doSetup self.retrieveAllMD() File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88, in retrieveAllMD dl = repo._async and repo._commonLoadRepoXML(repo) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1477, in _commonLoadRepoXML if self._latestRepoXML(local): File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1446, in _latestRepoXML oxml = self._saveOldRepoXML(local) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1303, in _saveOldRepoXML shutil.copy2(local, old_local) File "/usr/lib64/python2.7/shutil.py", line 130, in copy2 copyfile(src, dst) File "/usr/lib64/python2.7/shutil.py", line 83, in copyfile with open(dst, 'wb') as fdst: IOError: [Errno 28] No space left on device: '/var/cache/yum/x86_64/7/SCL-core/repomd.xml.old.tmp' /etc/cron.hourly/ghc-doc-index: /usr/bin/ghc-doc-index: line 27: /var/lib/ghc/pkg-dir.cache.new: No space left on device diff: /var/lib/ghc/pkg-dir.cache.new: No such file or directory haddock: internal error: .: copyFile: resource exhausted (No space left on device) From auton.sysnotify at gmail.com Sat Nov 3 15:50:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 15:50:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212106.67CDC1596247@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 16:00:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 16:00:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212106.7C6CA1596246@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 15:40:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 15:40:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212106.8282C1596247@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 15:20:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 15:20:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa1 1 1 Message-ID: <20181103212106.742761596243@gpu2.int.autonlab.org> Cannot write data to system activity file: No space left on device From auton.sysnotify at gmail.com Sat Nov 3 17:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 17:30:05 -0400 Subject: A desktop is down Message-ID: <20181103213006.5B10940F898E@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 18:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 18:00:06 -0400 Subject: A desktop is down Message-ID: <20181103220006.D005840F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 18:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 18:30:06 -0400 Subject: A desktop is down Message-ID: <20181103223006.CA03140F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 19:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 19:00:06 -0400 Subject: A desktop is down Message-ID: <20181103230006.A548840F898E@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 19:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 19:30:06 -0400 Subject: A desktop is down Message-ID: <20181103233006.4BB5340F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 20:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 20:00:05 -0400 Subject: A desktop is down Message-ID: <20181104000006.40C3240F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 20:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 20:30:06 -0400 Subject: A desktop is down Message-ID: <20181104003006.660DC40F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 21:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 21:00:05 -0400 Subject: A desktop is down Message-ID: <20181104010006.3E51640F898C@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 21:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 21:30:05 -0400 Subject: A desktop is down Message-ID: <20181104013006.495A440F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 22:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 22:00:06 -0400 Subject: A desktop is down Message-ID: <20181104020006.F3FD840F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 22:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 22:30:06 -0400 Subject: A desktop is down Message-ID: <20181104023006.9F1FA40F898A@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 23:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Sat, 03 Nov 2018 23:00:05 -0400 Subject: A desktop is down Message-ID: <20181104030006.58EC040F898F@lake.auton.cs.cmu.edu> luna.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 3 23:53:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 3 Nov 2018 23:53:01 -0400 (EDT) Subject: Cron /usr/lib64/sa/sa2 -A Message-ID: <20181104035301.DB52E101B2B@gpu2.int.autonlab.org> End of system activity file unexpected From auton.sysnotify at gmail.com Sun Nov 4 03:01:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Sun, 04 Nov 2018 08:01:01 -0000 Subject: neill-zfs.int.autonlab.org security run output Message-ID: <5bdea73e.1c69fb81.b46e9.ab3f@mx.google.com> neill-zfs.int.autonlab.org login failures: Nov 3 22:14:31 neill-zfs sshd[47770]: nss_ldap: ldap_start_tls failed: Timed out Nov 3 22:15:01 neill-zfs sshd[47770]: nss_ldap: ldap_start_tls failed: Timed out -- End of security output -- From auton.sysnotify at gmail.com Sun Nov 4 03:42:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Sun, 04 Nov 2018 03:42:25 -0500 Subject: SMART error (ErrorCount) detected on host: uranus Message-ID: <5bdeb0f1.35ac.51f88a27@uranus.int.autonlab.org> This message was generated by the smartd daemon running on: host name: uranus DNS domain: int.autonlab.org The following warning/error was logged by the smartd daemon: Device: /dev/da12 [SAT], ATA error count increased from 0 to 1 Device info: ST4000NM0024-1HT178, S/N:Z4F03FQ7, WWN:5-000c50-07b0cff9c, FW:SN02, 4.00 TB For details see host's SYSLOG. You can also use the smartctl utility for further investigation. No additional messages about this problem will be sent. From auton.sysnotify at gmail.com Sun Nov 4 22:00:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 04 Nov 2018 22:00:01 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 30 zroot/iocell/jails/600ec005-17f4-11e8-a693-0cc47aab0128/root Message-ID: <5bdfb231.127a.4625bc0f@warden.int.autonlab.org> There are 35 snapshots on 'zroot/iocell/jails/600ec005-17f4-11e8-a693-0cc47aab0128/root'. Can destroy 5 and keep 30. From auton.sysnotify at gmail.com Sun Nov 4 22:00:03 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 04 Nov 2018 22:00:03 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 50 zroot/iocell/jails/076a7105-0f6a-11e8-92df-0cc47a6baaa0/root Message-ID: <5bdfb233.39be.54c0b7ff@bhyve.int.autonlab.org> There are 59 snapshots on 'zroot/iocell/jails/076a7105-0f6a-11e8-92df-0cc47a6baaa0/root'. Can destroy 9 and keep 50. From auton.sysnotify at gmail.com Sun Nov 4 22:30:02 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 04 Nov 2018 22:30:02 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 30 zroot/iocell/jails/b4b45c86-c4ed-11e7-b09c-0cc47a6baaa0/root Message-ID: <5bdfb93a.127f.463165d9@warden.int.autonlab.org> There are 35 snapshots on 'zroot/iocell/jails/b4b45c86-c4ed-11e7-b09c-0cc47a6baaa0/root'. Can destroy 5 and keep 30. From auton.sysnotify at gmail.com Sun Nov 4 22:30:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 04 Nov 2018 22:30:01 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 50 zroot/iocell/jails/aea663c8-bf63-11e7-b4b4-0025900c0884/root Message-ID: <5bdfb939.39c2.36db00e@bhyve.int.autonlab.org> There are 59 snapshots on 'zroot/iocell/jails/aea663c8-bf63-11e7-b4b4-0025900c0884/root'. Can destroy 9 and keep 50. From auton.sysnotify at gmail.com Mon Nov 5 12:15:39 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Mon, 5 Nov 2018 12:15:39 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181105171539.319A02923D@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not get metalink https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64 error was 14: HTTPS Error 503 - Service Unavailable From auton.sysnotify at gmail.com Mon Nov 5 16:00:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Mon, 05 Nov 2018 21:00:09 -0000 Subject: Replication failed! Message-ID: <5be0af5a.1c69fb81.d1ed0.b0e7@mx.google.com> Hello, The replication failed for the local ZFS zfsneill/home because the command: /usr/bin/ssh -c arcfour256,arcfour128,blowfish-cbc,aes128-ctr,aes192-ctr,aes256-ctr -i /data/ssh/replication -o BatchMode=yes -o StrictHostKeyChecking=yes -o ConnectTimeout=7 have returned an error code of 255 From auton.sysnotify at gmail.com Mon Nov 5 16:30:08 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 05 Nov 2018 16:30:08 -0500 Subject: A desktop is down Message-ID: <20181105213010.87E6340F8993@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 5 17:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 05 Nov 2018 17:00:06 -0500 Subject: A desktop is down Message-ID: <20181105220006.7D88E40F8990@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 5 17:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 05 Nov 2018 17:30:06 -0500 Subject: A desktop is down Message-ID: <20181105223008.1048540F898C@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 7 07:03:19 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Wed, 07 Nov 2018 07:03:19 -0500 Subject: Cron /usr/local/sbin/iocell send -r -u root -h backup1 zroot Message-ID: <5be2d487.3b3a.1a60bebf@bhyve.int.autonlab.org> INFO: sending zroot/iocell to zroot/iocell: /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|done! From auton.sysnotify at gmail.com Fri Nov 9 15:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 09 Nov 2018 15:00:06 -0500 Subject: A desktop is down Message-ID: <20181109200006.862BF4127822@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Sat Nov 10 07:03:18 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 10 Nov 2018 07:03:18 -0500 Subject: Cron /usr/local/sbin/iocell send -r -u root -h uranus zroot Message-ID: <5be6c906.3d87.5a3fd25a@bhyve.int.autonlab.org> INFO: sending zroot/iocell to zroot/iocell: /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/done! From auton.sysnotify at gmail.com Sun Nov 11 03:01:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Sun, 11 Nov 2018 08:01:01 -0000 Subject: neill-zfs.int.autonlab.org security run output Message-ID: <5be7e1be.1c69fb81.ba7ef.9929@mx.google.com> neill-zfs.int.autonlab.org login failures: Nov 10 23:32:55 neill-zfs su: BAD SU auton to root on /dev/pts/0 -- End of security output -- From auton.sysnotify at gmail.com Sun Nov 11 22:00:02 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 11 Nov 2018 22:00:02 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 50 zroot/iocell/jails/076a7105-0f6a-11e8-92df-0cc47a6baaa0/root Message-ID: <5be8ecb2.3ee4.211fecd2@bhyve.int.autonlab.org> There are 59 snapshots on 'zroot/iocell/jails/076a7105-0f6a-11e8-92df-0cc47a6baaa0/root'. Can destroy 9 and keep 50. From auton.sysnotify at gmail.com Sun Nov 11 22:00:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 11 Nov 2018 22:00:01 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 30 zroot/iocell/jails/600ec005-17f4-11e8-a693-0cc47aab0128/root Message-ID: <5be8ecb1.1926.40c10f11@warden.int.autonlab.org> There are 35 snapshots on 'zroot/iocell/jails/600ec005-17f4-11e8-a693-0cc47aab0128/root'. Can destroy 5 and keep 30. From auton.sysnotify at gmail.com Sun Nov 11 22:30:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 11 Nov 2018 22:30:01 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 50 zroot/iocell/jails/aea663c8-bf63-11e7-b4b4-0025900c0884/root Message-ID: <5be8f3b9.3eec.68afd80c@bhyve.int.autonlab.org> There are 59 snapshots on 'zroot/iocell/jails/aea663c8-bf63-11e7-b4b4-0025900c0884/root'. Can destroy 9 and keep 50. From auton.sysnotify at gmail.com Sun Nov 11 22:30:02 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 11 Nov 2018 22:30:02 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 30 zroot/iocell/jails/b4b45c86-c4ed-11e7-b09c-0cc47a6baaa0/root Message-ID: <5be8f3ba.192a.9eebf2a@warden.int.autonlab.org> There are 35 snapshots on 'zroot/iocell/jails/b4b45c86-c4ed-11e7-b09c-0cc47a6baaa0/root'. Can destroy 5 and keep 30. From auton.sysnotify at gmail.com Tue Nov 13 12:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 13 Nov 2018 12:30:05 -0500 Subject: A desktop is down Message-ID: <20181113173006.5F55D412782D@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 13 13:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 13 Nov 2018 13:00:06 -0500 Subject: A desktop is down Message-ID: <20181113180006.B9C68412782D@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 13 13:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 13 Nov 2018 13:30:06 -0500 Subject: A desktop is down Message-ID: <20181113183006.964F5412782F@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 13 14:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 13 Nov 2018 14:00:05 -0500 Subject: A desktop is down Message-ID: <20181113190006.65669412782F@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 13 14:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 13 Nov 2018 14:30:06 -0500 Subject: A desktop is down Message-ID: <20181113193006.554DB412782D@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 13 15:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 13 Nov 2018 15:00:06 -0500 Subject: A desktop is down Message-ID: <20181113200006.4D6FF412782C@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 13 15:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 13 Nov 2018 15:30:06 -0500 Subject: A desktop is down Message-ID: <20181113203007.19A2F412782D@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 13 16:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 13 Nov 2018 16:00:06 -0500 Subject: A desktop is down Message-ID: <20181113210007.09060412782D@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 13 16:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 13 Nov 2018 16:30:06 -0500 Subject: A desktop is down Message-ID: <20181113213006.48838412782C@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 07:03:19 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Wed, 14 Nov 2018 07:03:19 -0500 Subject: Cron /usr/local/sbin/iocell send -r -u root -h backup1 zroot Message-ID: <5bec0f07.4083.7ec92517@bhyve.int.autonlab.org> INFO: sending zroot/iocell to zroot/iocell: /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-done! From auton.sysnotify at gmail.com Wed Nov 14 08:14:04 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Wed, 14 Nov 2018 08:14:04 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181114131404.966AD171ED@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not get metalink https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64 error was 14: curl#35 - "Encountered end of file" From auton.sysnotify at gmail.com Wed Nov 14 09:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 09:30:06 -0500 Subject: A desktop is down Message-ID: <20181114143007.08F064127822@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 10:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 10:00:06 -0500 Subject: A desktop is down Message-ID: <20181114150007.0D3384127822@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu lyra.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 10:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 10:30:05 -0500 Subject: A desktop is down Message-ID: <20181114153006.764EE4127825@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 11:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 11:00:05 -0500 Subject: A desktop is down Message-ID: <20181114160006.530134127825@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 11:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 11:30:06 -0500 Subject: A desktop is down Message-ID: <20181114163006.66BA6412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 12:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 12:00:05 -0500 Subject: A desktop is down Message-ID: <20181114170006.ADAE9412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 12:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 12:30:06 -0500 Subject: A desktop is down Message-ID: <20181114173007.4636B412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 13:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 13:00:06 -0500 Subject: A desktop is down Message-ID: <20181114180007.64610412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 13:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 13:30:06 -0500 Subject: A desktop is down Message-ID: <20181114183007.83CA4412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 14:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 14:00:06 -0500 Subject: A desktop is down Message-ID: <20181114190007.8D400412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 14:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 14:30:05 -0500 Subject: A desktop is down Message-ID: <20181114193006.8D983412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 14 15:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 14 Nov 2018 15:00:06 -0500 Subject: A desktop is down Message-ID: <20181114200007.3D886412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 06:02:30 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Thu, 15 Nov 2018 06:02:30 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bed5246.4119.6ea02d8e@bhyve.int.autonlab.org> ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Thu Nov 15 09:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 09:30:05 -0500 Subject: A desktop is down Message-ID: <20181115143006.5A776412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 10:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 10:00:06 -0500 Subject: A desktop is down Message-ID: <20181115150006.B92A9412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 10:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 10:30:05 -0500 Subject: A desktop is down Message-ID: <20181115153006.4A594412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 11:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 11:00:06 -0500 Subject: A desktop is down Message-ID: <20181115160006.975C1412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 11:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 11:30:06 -0500 Subject: A desktop is down Message-ID: <20181115163006.A562F412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 12:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 12:00:05 -0500 Subject: A desktop is down Message-ID: <20181115170006.32AF3412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 12:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 12:30:06 -0500 Subject: A desktop is down Message-ID: <20181115173006.8BBFE412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 13:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 13:00:05 -0500 Subject: A desktop is down Message-ID: <20181115180005.CA6D2412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 13:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 13:30:05 -0500 Subject: A desktop is down Message-ID: <20181115183006.1F146412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 14:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 14:00:06 -0500 Subject: A desktop is down Message-ID: <20181115190006.7BD97412AF72@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 14:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 14:30:06 -0500 Subject: A desktop is down Message-ID: <20181115193007.13694412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 15:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 15:00:06 -0500 Subject: A desktop is down Message-ID: <20181115200006.B0806412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 15:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 15:30:06 -0500 Subject: A desktop is down Message-ID: <20181115203006.88CAC412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 15 16:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 15 Nov 2018 16:00:05 -0500 Subject: A desktop is down Message-ID: <20181115210006.0CE86412AF71@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 03:01:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Fri, 16 Nov 2018 08:01:01 -0000 Subject: neill-zfs.int.autonlab.org security run output Message-ID: <5bee793e.1c69fb81.855c1.adc5@mx.google.com> neill-zfs.int.autonlab.org login failures: Nov 15 22:50:02 neill-zfs sshd[75383]: Failed password for root from 10.8.0.6 port 32898 ssh2 -- End of security output -- From auton.sysnotify at gmail.com Fri Nov 16 06:02:30 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Fri, 16 Nov 2018 06:02:30 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5beea3c6.41b6.74cde686@bhyve.int.autonlab.org> ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Fri Nov 16 11:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 16 Nov 2018 11:00:05 -0500 Subject: A desktop is down Message-ID: <20181116160006.5269D412AF71@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 11:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 16 Nov 2018 11:30:05 -0500 Subject: A desktop is down Message-ID: <20181116163006.760AE412AF71@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 12:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 16 Nov 2018 12:00:06 -0500 Subject: A desktop is down Message-ID: <20181116170007.17939412AF71@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 12:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 16 Nov 2018 12:30:06 -0500 Subject: A desktop is down Message-ID: <20181116173006.7D95C412AF71@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 13:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 16 Nov 2018 13:00:06 -0500 Subject: A desktop is down Message-ID: <20181116180006.F25C1412AF71@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 13:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 16 Nov 2018 13:30:05 -0500 Subject: A desktop is down Message-ID: <20181116183006.732A0412AF71@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 14:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 16 Nov 2018 14:00:06 -0500 Subject: A desktop is down Message-ID: <20181116190007.11668412AF71@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 14:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 16 Nov 2018 14:30:06 -0500 Subject: A desktop is down Message-ID: <20181116193006.78526412AF71@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 15:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 16 Nov 2018 15:00:06 -0500 Subject: A desktop is down Message-ID: <20181116200007.11119412AF71@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 15:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 16 Nov 2018 15:30:06 -0500 Subject: A desktop is down Message-ID: <20181116203007.1EE38412AF71@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 16:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 16 Nov 2018 16:00:06 -0500 Subject: A desktop is down Message-ID: <20181116210006.851EA412AF71@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 16 20:07:10 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Fri, 16 Nov 2018 20:07:10 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181117010710.E1021C9021@ari.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not get metalink https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64 error was 14: HTTPS Error 503 - Service Unavailable From auton.sysnotify at gmail.com Sat Nov 17 06:02:30 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 17 Nov 2018 06:02:30 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5beff546.425b.46b414c2@bhyve.int.autonlab.org> ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Sat Nov 17 07:03:19 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 17 Nov 2018 07:03:19 -0500 Subject: Cron /usr/local/sbin/iocell send -r -u root -h uranus zroot Message-ID: <5bf00387.4264.1bdaa9fb@bhyve.int.autonlab.org> INFO: sending zroot/iocell to zroot/iocell: /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|done! From auton.sysnotify at gmail.com Sat Nov 17 08:00:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Sat, 17 Nov 2018 08:00:47 -0500 Subject: SMART error (OfflineUncorrectableSector) detected on host: uranus Message-ID: <5bf010ff.c30.56b71544@uranus.int.autonlab.org> This message was generated by the smartd daemon running on: host name: uranus DNS domain: int.autonlab.org The following warning/error was logged by the smartd daemon: Device: /dev/da12 [SAT], 8 Offline uncorrectable sectors Device info: ST4000NM0024-1HT178, S/N:Z4F03FQ7, WWN:5-000c50-07b0cff9c, FW:SN02, 4.00 TB For details see host's SYSLOG. You can also use the smartctl utility for further investigation. No additional messages about this problem will be sent. From auton.sysnotify at gmail.com Sat Nov 17 08:00:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Sat, 17 Nov 2018 08:00:47 -0500 Subject: SMART error (SelfTest) detected on host: uranus Message-ID: <5bf010ff.c32.28240d30@uranus.int.autonlab.org> This message was generated by the smartd daemon running on: host name: uranus DNS domain: int.autonlab.org The following warning/error was logged by the smartd daemon: Device: /dev/da12 [SAT], Self-Test Log error count increased from 0 to 1 Device info: ST4000NM0024-1HT178, S/N:Z4F03FQ7, WWN:5-000c50-07b0cff9c, FW:SN02, 4.00 TB For details see host's SYSLOG. You can also use the smartctl utility for further investigation. No additional messages about this problem will be sent. From auton.sysnotify at gmail.com Sat Nov 17 08:00:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Sat, 17 Nov 2018 08:00:47 -0500 Subject: SMART error (CurrentPendingSector) detected on host: uranus Message-ID: <5bf010ff.c2e.c4d858f@uranus.int.autonlab.org> This message was generated by the smartd daemon running on: host name: uranus DNS domain: int.autonlab.org The following warning/error was logged by the smartd daemon: Device: /dev/da12 [SAT], 8 Currently unreadable (pending) sectors Device info: ST4000NM0024-1HT178, S/N:Z4F03FQ7, WWN:5-000c50-07b0cff9c, FW:SN02, 4.00 TB For details see host's SYSLOG. You can also use the smartctl utility for further investigation. No additional messages about this problem will be sent. From auton.sysnotify at gmail.com Sat Nov 17 11:13:16 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 17 Nov 2018 11:13:16 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181117161316.8F155D9E6@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Sun Nov 18 00:00:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Sun, 18 Nov 2018 05:00:05 -0000 Subject: Cron PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/root/bin" /usr/local/sbin/scrub -t 35 zfsneill Message-ID: <5bf0f1d6.1c69fb81.8f99f.c0fd@mx.google.com> starting scrub of pool 'zfsneill' From auton.sysnotify at gmail.com Sun Nov 18 03:09:38 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 18 Nov 2018 03:09:38 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181118080938.9C482101B42@gpu2.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Sun Nov 18 06:02:30 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 18 Nov 2018 06:02:30 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bf146c6.42fd.48bb2be7@bhyve.int.autonlab.org> ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Sun Nov 18 07:13:47 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 18 Nov 2018 07:13:47 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181118121349.CAFF7763@ari.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Sun Nov 18 12:10:47 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 18 Nov 2018 12:10:47 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181118171048.1CC7029C80@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Sun Nov 18 14:30:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Sun, 18 Nov 2018 14:30:46 -0500 Subject: SMART error (ErrorCount) detected on host: uranus Message-ID: <5bf1bde6.10a5.4603d3b9@uranus.int.autonlab.org> This message was generated by the smartd daemon running on: host name: uranus DNS domain: int.autonlab.org The following warning/error was logged by the smartd daemon: Device: /dev/da12 [SAT], ATA error count increased from 1 to 2 Device info: ST4000NM0024-1HT178, S/N:Z4F03FQ7, WWN:5-000c50-07b0cff9c, FW:SN02, 4.00 TB For details see host's SYSLOG. You can also use the smartctl utility for further investigation. No additional messages about this problem will be sent. From auton.sysnotify at gmail.com Sun Nov 18 22:00:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 18 Nov 2018 22:00:01 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 30 zroot/iocell/jails/600ec005-17f4-11e8-a693-0cc47aab0128/root Message-ID: <5bf22731.2cd9.5bf1048b@warden.int.autonlab.org> There are 35 snapshots on 'zroot/iocell/jails/600ec005-17f4-11e8-a693-0cc47aab0128/root'. Can destroy 5 and keep 30. From auton.sysnotify at gmail.com Sun Nov 18 22:00:02 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 18 Nov 2018 22:00:02 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 50 zroot/iocell/jails/076a7105-0f6a-11e8-92df-0cc47a6baaa0/root Message-ID: <5bf22732.4363.59dd8f1f@bhyve.int.autonlab.org> There are 59 snapshots on 'zroot/iocell/jails/076a7105-0f6a-11e8-92df-0cc47a6baaa0/root'. Can destroy 9 and keep 50. From auton.sysnotify at gmail.com Sun Nov 18 22:30:02 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 18 Nov 2018 22:30:02 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 30 zroot/iocell/jails/b4b45c86-c4ed-11e7-b09c-0cc47a6baaa0/root Message-ID: <5bf22e3a.2cdd.6c22d26d@warden.int.autonlab.org> There are 35 snapshots on 'zroot/iocell/jails/b4b45c86-c4ed-11e7-b09c-0cc47a6baaa0/root'. Can destroy 5 and keep 30. From auton.sysnotify at gmail.com Sun Nov 18 22:30:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 18 Nov 2018 22:30:01 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 50 zroot/iocell/jails/aea663c8-bf63-11e7-b4b4-0025900c0884/root Message-ID: <5bf22e39.4367.49577105@bhyve.int.autonlab.org> There are 59 snapshots on 'zroot/iocell/jails/aea663c8-bf63-11e7-b4b4-0025900c0884/root'. Can destroy 9 and keep 50. From auton.sysnotify at gmail.com Mon Nov 19 06:02:30 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Mon, 19 Nov 2018 06:02:30 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bf29846.439d.5142a8d5@bhyve.int.autonlab.org> ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Mon Nov 19 08:03:28 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Mon, 19 Nov 2018 08:03:28 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181119130328.CAFB4498@ari.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Mon Nov 19 11:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 11:00:06 -0500 Subject: A desktop is down Message-ID: <20181119160007.45F70412B48B@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 19 11:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 11:30:06 -0500 Subject: A desktop is down Message-ID: <20181119163007.26BEB412B48B@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 19 12:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 12:00:06 -0500 Subject: A desktop is down Message-ID: <20181119170007.434B0412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 19 12:16:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Mon, 19 Nov 2018 12:16:00 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181119171601.12EBD4AB30A@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Mon Nov 19 12:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 12:30:05 -0500 Subject: A desktop is down Message-ID: <20181119173006.315E2412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 19 13:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 13:00:05 -0500 Subject: A desktop is down Message-ID: <20181119180005.E11F4412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 19 13:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 13:30:06 -0500 Subject: A desktop is down Message-ID: <20181119183007.0AEDD412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 19 14:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 14:00:06 -0500 Subject: A desktop is down Message-ID: <20181119190006.DDB95412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 19 14:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 14:30:06 -0500 Subject: A desktop is down Message-ID: <20181119193006.80BC5412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 19 15:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 15:00:06 -0500 Subject: A desktop is down Message-ID: <20181119200006.45FDB412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 19 15:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 15:30:06 -0500 Subject: A desktop is down Message-ID: <20181119203006.EFF8D412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 19 16:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 16:00:06 -0500 Subject: A desktop is down Message-ID: <20181119210006.B909E412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 19 16:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 19 Nov 2018 16:30:05 -0500 Subject: A desktop is down Message-ID: <20181119213006.795B6412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 04:05:09 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Tue, 20 Nov 2018 04:05:09 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181120090509.D9DB1101B43@gpu2.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Tue Nov 20 06:02:30 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Tue, 20 Nov 2018 06:02:30 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bf3e9c6.443d.4b1fa2ea@bhyve.int.autonlab.org> ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out ssh: connect to host backup1.int.autonlab.org port 22: Operation timed out Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Tue Nov 20 08:08:19 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Tue, 20 Nov 2018 08:08:19 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181120130820.28982498@ari.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Tue Nov 20 09:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 09:30:06 -0500 Subject: A desktop is down Message-ID: <20181120143007.74AEE412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 10:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 10:00:05 -0500 Subject: A desktop is down Message-ID: <20181120150005.DA3F8412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 10:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 10:30:06 -0500 Subject: A desktop is down Message-ID: <20181120153006.961DB412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 11:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 11:00:05 -0500 Subject: A desktop is down Message-ID: <20181120160006.161E4412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 11:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 11:30:06 -0500 Subject: A desktop is down Message-ID: <20181120163007.089A4412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 12:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 12:00:06 -0500 Subject: A desktop is down Message-ID: <20181120170007.16640412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 12:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 12:30:06 -0500 Subject: A desktop is down Message-ID: <20181120173006.AB85D412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 13:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 13:00:06 -0500 Subject: A desktop is down Message-ID: <20181120180006.C2CED412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 13:11:08 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Tue, 20 Nov 2018 13:11:08 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181120181108.774124AB30B@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Tue Nov 20 13:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 13:30:06 -0500 Subject: A desktop is down Message-ID: <20181120183006.86946412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 14:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 14:00:05 -0500 Subject: A desktop is down Message-ID: <20181120190005.F24B2412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 14:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 14:30:06 -0500 Subject: A desktop is down Message-ID: <20181120193006.BB060412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 15:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 15:00:06 -0500 Subject: A desktop is down Message-ID: <20181120200006.78FB4412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 15:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 15:30:06 -0500 Subject: A desktop is down Message-ID: <20181120203006.97E14412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 16:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 16:00:05 -0500 Subject: A desktop is down Message-ID: <20181120210006.1C1C4412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 20 16:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 20 Nov 2018 16:30:06 -0500 Subject: A desktop is down Message-ID: <20181120213007.1DAD6412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 04:07:24 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Wed, 21 Nov 2018 04:07:24 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181121090724.F2367101B36@gpu2.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Wed Nov 21 06:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Wed, 21 Nov 2018 06:00:00 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bf53ab0.44db.433831a8@bhyve.int.autonlab.org> ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Wed Nov 21 07:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Wed, 21 Nov 2018 07:00:00 -0500 Subject: Cron /usr/local/sbin/iocell send -r -u root -h backup1 zroot Message-ID: <5bf548c0.44e5.7f9f0849@bhyve.int.autonlab.org> ERROR: please set up publickey authentication for root at backup1 From auton.sysnotify at gmail.com Wed Nov 21 08:15:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Wed, 21 Nov 2018 08:15:01 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181121131501.CBC23175E6B@ari.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Wed Nov 21 10:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 10:00:06 -0500 Subject: A desktop is down Message-ID: <20181121150007.56F76412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 10:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 10:30:06 -0500 Subject: A desktop is down Message-ID: <20181121153006.9F7A9412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 11:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 11:00:06 -0500 Subject: A desktop is down Message-ID: <20181121160006.80539412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 11:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 11:30:06 -0500 Subject: A desktop is down Message-ID: <20181121163007.0FC75412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 12:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 12:00:06 -0500 Subject: A desktop is down Message-ID: <20181121170006.DAC6A412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 12:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 12:30:06 -0500 Subject: A desktop is down Message-ID: <20181121173006.61C05412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 13:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 13:00:05 -0500 Subject: A desktop is down Message-ID: <20181121180006.02EA9412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 13:12:20 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Wed, 21 Nov 2018 13:12:20 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181121181220.DC8814AB30D@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Wed Nov 21 13:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 13:30:06 -0500 Subject: A desktop is down Message-ID: <20181121183006.2B044412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 14:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 14:00:06 -0500 Subject: A desktop is down Message-ID: <20181121190007.00A99412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 14:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 14:30:06 -0500 Subject: A desktop is down Message-ID: <20181121193006.CC86D412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 15:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 15:00:06 -0500 Subject: A desktop is down Message-ID: <20181121200006.85655412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 15:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 15:30:06 -0500 Subject: A desktop is down Message-ID: <20181121203006.DD3A5412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 21 16:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 21 Nov 2018 16:00:06 -0500 Subject: A desktop is down Message-ID: <20181121210006.615CE412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 22 01:28:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:28:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64c8e.Kr9QRpOeHrOYkhjD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 1 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868021 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:28:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:28:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64c91.VNoSRQlxYqB1PI4Z%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 2 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868103 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:28:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:28:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64c9c.1oqdN90aiFzNXXMP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 3 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868124 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:29:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:29:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64cb5.iYPZBKNhmPyQlap8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 4 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868148 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:29:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:29:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64cce.CDtHnZc5oVnWzwf7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 5 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868173 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:29:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:29:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64ce6.T4gg+OyGZWojU7+T%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 6 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868197 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:30:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:30:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64cfe.wg1DdQoFTKA850fF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 7 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868222 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:30:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:30:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64d16.996AwDDle/sGNk4h%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 8 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868246 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:31:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:31:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64d2f.h74ExbL/SK0JCsX7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 9 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868270 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:31:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:31:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64d48.79N6fx/RCJlb7YHO%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 10 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868295 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:32:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:32:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64d60.7icJZLijDvTHMk8C%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 11 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868320 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:32:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:32:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64d77.Ur8PAqsXoJdCtTJE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 12 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868343 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:32:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:32:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64d90.GFcpFm4tKQkD/C+w%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 13 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868368 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:33:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:33:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64da9.dguc4umz8sVM7wx5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 14 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868392 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:33:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:33:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64dc2.J8NR8f3lQeRDKvM9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 15 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868418 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:34:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:34:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64ddb.qbDNcYTzJ0Ma/cLr%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 16 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868442 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:34:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:34:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64df2.XsCiWvBcmDrqRdSg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 17 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868466 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:34:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:34:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64e0a.OhsOySJlBPvzyTJD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 18 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868490 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:35:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:35:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64e23.aQizpKm72RZ3MrSU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 19 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868515 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:35:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:35:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64e3b.y0GxXttqD/q8ZoMQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 20 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868539 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:36:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:36:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64e54.V/IdBK5UA+70t6GH%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 21 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868564 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:36:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:36:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64e6d.WDmAWqTYwzhKncaM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 22 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868589 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 01:36:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 01:36:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf64e85.X085cDSOEaomWAfR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 7107 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu1.int.autonlab.org count: 23 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 7107 kernel: 3.10.0-862.14.4.el7.x86_64 last_occurrence: 1542868613 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 01:27:37 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 139752245465312 : , "build_id_offset": 12512 : } : , { "address": 139752271338345 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 139752271338423 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204477 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10173 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:net_prio,net_cls:/ :10:cpuset:/ :9:pids:/user.slice :8:blkio:/ :7:hugetlb:/ :6:freezer:/ :5:devices:/user.slice :4:memory:/ :3:cpuacct,cpu:/ :2:perf_event:/ :1:name=systemd:/user.slice/user-1143.slice/session-2905.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=2905 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu1.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 32518 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/9 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,21782,0 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%3 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 32518 192.168.6.100 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESH/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202265760 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202265760 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202265760 /usr/bin/watch :011b0000-01277000 rw-p 00000000 00:00 0 [heap] :7f1a93d3b000-7f1a9a265000 r--p 00000000 fd:00 201389418 /usr/lib/locale/locale-archive :7f1a9a265000-7f1a9a274000 r-xp 00000000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a274000-7f1a9a473000 ---p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a473000-7f1a9a474000 r--p 0000e000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a474000-7f1a9a475000 rw-p 0000f000 fd:00 60808 /usr/lib64/libbz2.so.1.0.6 :7f1a9a475000-7f1a9a48a000 r-xp 00000000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a48a000-7f1a9a689000 ---p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a689000-7f1a9a68a000 r--p 00014000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68a000-7f1a9a68b000 rw-p 00015000 fd:00 97935 /usr/lib64/libz.so.1.2.7 :7f1a9a68b000-7f1a9a6a2000 r-xp 00000000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a6a2000-7f1a9a8a1000 ---p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a1000-7f1a9a8a2000 r--p 00016000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a2000-7f1a9a8a3000 rw-p 00017000 fd:00 1818716 /usr/lib64/libelf-0.172.so :7f1a9a8a3000-7f1a9a903000 r-xp 00000000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9a903000-7f1a9ab03000 ---p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab03000-7f1a9ab04000 r--p 00060000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab04000-7f1a9ab05000 rw-p 00061000 fd:00 58049 /usr/lib64/libpcre.so.1.2.0 :7f1a9ab05000-7f1a9ab09000 r-xp 00000000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ab09000-7f1a9ad08000 ---p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad08000-7f1a9ad09000 r--p 00003000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad09000-7f1a9ad0a000 rw-p 00004000 fd:00 60825 /usr/lib64/libattr.so.1.1.0 :7f1a9ad0a000-7f1a9ad21000 r-xp 00000000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9ad21000-7f1a9af20000 ---p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af20000-7f1a9af21000 r--p 00016000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af21000-7f1a9af22000 rw-p 00017000 fd:00 53499 /usr/lib64/libpthread-2.17.so :7f1a9af22000-7f1a9af26000 rw-p 00000000 00:00 0 :7f1a9af26000-7f1a9af3c000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9af3c000-7f1a9b13b000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13b000-7f1a9b13c000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13c000-7f1a9b13d000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7f1a9b13d000-7f1a9b189000 r-xp 00000000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b189000-7f1a9b389000 ---p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b389000-7f1a9b38b000 r--p 0004c000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38b000-7f1a9b38c000 rw-p 0004e000 fd:00 1715213 /usr/lib64/libdw-0.172.so :7f1a9b38c000-7f1a9b3a2000 r-xp 00000000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b3a2000-7f1a9b5a1000 ---p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a1000-7f1a9b5a2000 r--p 00015000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a2000-7f1a9b5a3000 rw-p 00016000 fd:00 53501 /usr/lib64/libresolv-2.17.so :7f1a9b5a3000-7f1a9b5a5000 rw-p 00000000 00:00 0 :7f1a9b5a5000-7f1a9b5a9000 r-xp 00000000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b5a9000-7f1a9b7a8000 ---p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a8000-7f1a9b7a9000 r--p 00003000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7a9000-7f1a9b7aa000 rw-p 00004000 fd:00 63522 /usr/lib64/libgpg-error.so.0.10.0 :7f1a9b7aa000-7f1a9b827000 r-xp 00000000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9b827000-7f1a9ba26000 ---p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba26000-7f1a9ba27000 r--p 0007c000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba27000-7f1a9ba2a000 rw-p 0007d000 fd:00 63525 /usr/lib64/libgcrypt.so.11.8.2 :7f1a9ba2a000-7f1a9ba2b000 rw-p 00000000 00:00 0 :7f1a9ba2b000-7f1a9ba3f000 r-xp 00000000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9ba3f000-7f1a9bc3e000 ---p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3e000-7f1a9bc3f000 r--p 00013000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc3f000-7f1a9bc40000 rw-p 00014000 fd:00 1709632 /usr/lib64/liblz4.so.1.7.5 :7f1a9bc40000-7f1a9bc65000 r-xp 00000000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9bc65000-7f1a9be64000 ---p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be64000-7f1a9be65000 r--p 00024000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be65000-7f1a9be66000 rw-p 00025000 fd:00 60780 /usr/lib64/liblzma.so.5.2.2 :7f1a9be66000-7f1a9be8a000 r-xp 00000000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9be8a000-7f1a9c089000 ---p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c089000-7f1a9c08a000 r--p 00023000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08a000-7f1a9c08b000 rw-p 00024000 fd:00 53516 /usr/lib64/libselinux.so.1 :7f1a9c08b000-7f1a9c08d000 rw-p 00000000 00:00 0 :7f1a9c08d000-7f1a9c094000 r-xp 00000000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c094000-7f1a9c293000 ---p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c293000-7f1a9c294000 r--p 00006000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c294000-7f1a9c295000 rw-p 00007000 fd:00 53503 /usr/lib64/librt-2.17.so :7f1a9c295000-7f1a9c396000 r-xp 00000000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c396000-7f1a9c595000 ---p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c595000-7f1a9c596000 r--p 00100000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c596000-7f1a9c597000 rw-p 00101000 fd:00 53481 /usr/lib64/libm-2.17.so :7f1a9c597000-7f1a9c59b000 r-xp 00000000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c59b000-7f1a9c79a000 ---p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79a000-7f1a9c79b000 r--p 00003000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79b000-7f1a9c79c000 rw-p 00004000 fd:00 60829 /usr/lib64/libcap.so.2.22 :7f1a9c79c000-7f1a9c95e000 r-xp 00000000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9c95e000-7f1a9cb5e000 ---p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb5e000-7f1a9cb62000 r--p 001c2000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb62000-7f1a9cb64000 rw-p 001c6000 fd:00 53473 /usr/lib64/libc-2.17.so :7f1a9cb64000-7f1a9cb69000 rw-p 00000000 00:00 0 :7f1a9cb69000-7f1a9cb6b000 r-xp 00000000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cb6b000-7f1a9cd6b000 ---p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6b000-7f1a9cd6c000 r--p 00002000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6c000-7f1a9cd6d000 rw-p 00003000 fd:00 53479 /usr/lib64/libdl-2.17.so :7f1a9cd6d000-7f1a9cd92000 r-xp 00000000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cd92000-7f1a9cf92000 ---p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf92000-7f1a9cf96000 r--p 00025000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf96000-7f1a9cf97000 rw-p 00029000 fd:00 319106 /usr/lib64/libtinfo.so.5.9 :7f1a9cf97000-7f1a9cfce000 r-xp 00000000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9cfce000-7f1a9d1cd000 ---p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cd000-7f1a9d1ce000 r--p 00036000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1ce000-7f1a9d1cf000 rw-p 00037000 fd:00 58111 /usr/lib64/libncursesw.so.5.9 :7f1a9d1cf000-7f1a9d1fe000 r-xp 00000000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d1fe000-7f1a9d3fe000 ---p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3fe000-7f1a9d3ff000 r--p 0002f000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d3ff000-7f1a9d400000 rw-p 00030000 fd:00 1818692 /usr/lib64/libsystemd.so.0.6.0 :7f1a9d400000-7f1a9d411000 r-xp 00000000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d411000-7f1a9d611000 ---p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d611000-7f1a9d612000 r--p 00011000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d612000-7f1a9d613000 rw-p 00012000 fd:00 242257 /usr/lib64/libprocps.so.4.0.0 :7f1a9d613000-7f1a9d627000 rw-p 00000000 00:00 0 :7f1a9d627000-7f1a9d649000 r-xp 00000000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d818000-7f1a9d825000 rw-p 00000000 00:00 0 :7f1a9d840000-7f1a9d847000 r--s 00000000 fd:00 67644279 /usr/lib64/gconv/gconv-modules.cache :7f1a9d847000-7f1a9d848000 rw-p 00000000 00:00 0 :7f1a9d848000-7f1a9d849000 r--p 00021000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d849000-7f1a9d84a000 rw-p 00022000 fd:00 22196685 /usr/lib64/ld-2.17.so :7f1a9d84a000-7f1a9d84b000 rw-p 00000000 00:00 0 :7ffcfceee000-7ffcfcf10000 rw-p 00000000 00:00 0 [stack] :7ffcfcfce000-7ffcfcfd0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/13 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 7107 :Ngid: 0 :Pid: 7107 :PPid: 22106 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1436 kB :VmRSS: 992 kB :RssAnon: 980 kB :RssFile: 12 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 136 kB :VmSwap: 76 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Speculation_Store_Bypass: thread vulnerable :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 27 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:42:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:42:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65dce.LgFegT6mImpfsTPJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 1481 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 1481 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872523 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:42:06 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 1481 :Ngid: 0 :Pid: 1481 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:42:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:42:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65dcc.ZWMUwdso3V9qanA5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 1450 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 1450 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872522 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:42:03 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 1450 :Ngid: 0 :Pid: 1450 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e4a.hbRfC2XhDCT/eipx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 1552 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 1552 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872526 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:42:07 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 1552 :Ngid: 0 :Pid: 1552 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e4c.D2M6LwYBaByXRmpC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 1641 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 1641 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872529 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:42:10 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 1641 :Ngid: 0 :Pid: 1641 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e4b.ZEiXofqL9U+YLJ0m%user@localhost> reason: watch killed by SIGBUS cmdline: watch nvidia-smi executable: /usr/bin/watch pid: 1597 pwd: /zfsauton/home/bpatra/MUWE (deleted) hostname: gpu2.int.autonlab.org count: 1 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 1597 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872527 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:42:08 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140525775360224 : , "build_id_offset": 12512 : } : , { "address": 140525801233257 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140525801233335 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,3 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/MUWE :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%23 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=10.8.0.6 51754 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESHOME=config :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :01255000-013a4000 rw-p 00000000 00:00 0 [heap] :7fceadcd3000-7fceb41fd000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fceb41fd000-7fceb420c000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fceb420c000-7fceb440b000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fceb440b000-7fceb440c000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fceb440c000-7fceb440d000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fceb440d000-7fceb4422000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fceb4422000-7fceb4621000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fceb4621000-7fceb4622000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fceb4622000-7fceb4623000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fceb4623000-7fceb463a000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fceb463a000-7fceb4839000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fceb4839000-7fceb483a000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fceb483a000-7fceb483b000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fceb483b000-7fceb489b000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fceb489b000-7fceb4a9b000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fceb4a9b000-7fceb4a9c000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fceb4a9c000-7fceb4a9d000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fceb4a9d000-7fceb4aa1000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fceb4aa1000-7fceb4ca0000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fceb4ca0000-7fceb4ca1000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fceb4ca1000-7fceb4ca2000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fceb4ca2000-7fceb4cb9000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fceb4cb9000-7fceb4eb8000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fceb4eb8000-7fceb4eb9000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fceb4eb9000-7fceb4eba000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fceb4eba000-7fceb4ebe000 rw-p 00000000 00:00 0 :7fceb4ebe000-7fceb4ed4000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fceb4ed4000-7fceb50d3000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fceb50d3000-7fceb50d4000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fceb50d4000-7fceb50d5000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fceb50d5000-7fceb5121000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fceb5121000-7fceb5321000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fceb5321000-7fceb5323000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fceb5323000-7fceb5324000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fceb5324000-7fceb533a000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fceb533a000-7fceb5539000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fceb5539000-7fceb553a000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fceb553a000-7fceb553b000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fceb553b000-7fceb553d000 rw-p 00000000 00:00 0 :7fceb553d000-7fceb5541000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fceb5541000-7fceb5740000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fceb5740000-7fceb5741000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fceb5741000-7fceb5742000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fceb5742000-7fceb57bf000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fceb57bf000-7fceb59be000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fceb59be000-7fceb59bf000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fceb59bf000-7fceb59c2000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fceb59c2000-7fceb59c3000 rw-p 00000000 00:00 0 :7fceb59c3000-7fceb59d7000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fceb59d7000-7fceb5bd6000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fceb5bd6000-7fceb5bd7000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fceb5bd7000-7fceb5bd8000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fceb5bd8000-7fceb5bfd000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fceb5bfd000-7fceb5dfc000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fceb5dfc000-7fceb5dfd000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fceb5dfd000-7fceb5dfe000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fceb5dfe000-7fceb5e22000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fceb5e22000-7fceb6021000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fceb6021000-7fceb6022000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fceb6022000-7fceb6023000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fceb6023000-7fceb6025000 rw-p 00000000 00:00 0 :7fceb6025000-7fceb602c000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fceb602c000-7fceb622b000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fceb622b000-7fceb622c000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fceb622c000-7fceb622d000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fceb622d000-7fceb632e000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fceb632e000-7fceb652d000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fceb652d000-7fceb652e000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fceb652e000-7fceb652f000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fceb652f000-7fceb6533000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fceb6533000-7fceb6732000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fceb6732000-7fceb6733000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fceb6733000-7fceb6734000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fceb6734000-7fceb68f6000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fceb68f6000-7fceb6af6000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fceb6af6000-7fceb6afa000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fceb6afa000-7fceb6afc000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fceb6afc000-7fceb6b01000 rw-p 00000000 00:00 0 :7fceb6b01000-7fceb6b03000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fceb6b03000-7fceb6d03000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fceb6d03000-7fceb6d04000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fceb6d04000-7fceb6d05000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fceb6d05000-7fceb6d2a000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fceb6d2a000-7fceb6f2a000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fceb6f2a000-7fceb6f2e000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fceb6f2e000-7fceb6f2f000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fceb6f2f000-7fceb6f66000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fceb6f66000-7fceb7165000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fceb7165000-7fceb7166000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fceb7166000-7fceb7167000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fceb7167000-7fceb7196000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fceb7196000-7fceb7396000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fceb7396000-7fceb7397000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fceb7397000-7fceb7398000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fceb7398000-7fceb73a9000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fceb73a9000-7fceb75a9000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fceb75a9000-7fceb75aa000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fceb75aa000-7fceb75ab000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fceb75ab000-7fceb75bf000 rw-p 00000000 00:00 0 :7fceb75bf000-7fceb75e1000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fceb77ac000-7fceb77b9000 rw-p 00000000 00:00 0 :7fceb77d8000-7fceb77df000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fceb77df000-7fceb77e0000 rw-p 00000000 00:00 0 :7fceb77e0000-7fceb77e1000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fceb77e1000-7fceb77e2000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fceb77e2000-7fceb77e3000 rw-p 00000000 00:00 0 :7fff5fd73000-7fff5fd95000 rw-p 00000000 00:00 0 [stack] :7fff5fdb9000-7fff5fdbb000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/44 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 1597 :Ngid: 0 :Pid: 1597 :PPid: 10298 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 158336 kB :VmSize: 158336 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1248 kB :VmRSS: 1248 kB :RssAnon: 816 kB :RssFile: 432 kB :RssShmem: 0 kB :VmData: 1544 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 128 kB :VmSwap: 788 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e4b.PZm3EbKTcbxsaiOh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 1609 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 1609 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872528 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:42:09 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 1609 :Ngid: 0 :Pid: 1609 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e4e.LuH5mJyBXVRa5pgX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872530 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e4e.nlkeKU8y7iYftKRl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872531 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e4d.zdjYZIljxdUVPmTS%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872530 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e51.K7tTWlm4V7z0P85D%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872534 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e52.a8HCuDKtZjhrqPoU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872652 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e4f.7WGbabc2KNRloATY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872533 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e50.UeudNYAiNodjZBke%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872533 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e53.ynHHKVVdH+xrxxnm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872654 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e54.fxuOkWH/QfgWktTV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 9 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872655 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e58.H8Zz/eRn2uqpol4n%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872656 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e5a.z0VdyG3nrU17g6dF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 11 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872657 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e5b.I04EIP0CG2gRiAeh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872658 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e5d.Yd8hqRfso3idUfoc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 14 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872668 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e5c.ZKYJiu/s2ZUUiwNh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872661 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e5e.H8xPjLoDLUCgBuU6%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 16 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872669 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e5e.ZetjAC66VEL6AQ+f%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 15 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872668 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e5f.L8+a/ELDjLbj52La%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 17 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872671 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e60.g2M4wcLg0fOA2E6v%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 18 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872672 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e62.BAG6ZgKQLUdEJsJp%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 19 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872673 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e63.fnOhPCQ6RzpMoWtG%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 20 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872675 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e64.HwM9JXyeE0hLJVzc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 21 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872676 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:37 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e65.0x74MhYRQLCalpaP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 22 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872676 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e66.95bssgH9ySzeJAuY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 23 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872678 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e67.PUpaeDfdhjTmPmh3%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 24 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872678 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e6a.+L1pnGoW0GBGOHRb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 27 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872682 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e68.DgWkphXCTESjj6Fm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 25 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872679 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e69.qeCgFaIyCJQtaMER%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 26 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872680 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e6c.tGZ+t/pm2Vh+Yq4k%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 29 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872683 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e6b.y6vRMvNRM9SoQYpX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 28 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872683 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e6d.p3PR9rBPzgJWQGzn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 30 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872685 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e6e.7qD/toSIz+46+cf9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 31 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872685 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e6e.d7n9EZh/CWdu+iZm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 32 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872686 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e70.4fqKjjVINNfXcGKU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 33 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872687 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e71.NGzch5+5J745CaVA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 34 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872689 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e73.kEExZJf9mF72/2+k%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 35 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872690 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e74.J4QPtXr55GU2AF9C%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 36 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872691 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e75.g6FKutGcBVXRbHli%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 37 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872692 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e76.zRJ60nNxPzNLfB55%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 38 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872694 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e77.dE6PVzIo8TiYOcha%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 39 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872694 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e77.MEarMwuhCFtL+LsP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 40 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872695 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e78.9ADxNfZJ3XIqyWhy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 41 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872696 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e7d.bDO7fZ3HHMbDIwbU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 46 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872700 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e7b.2Ys5IiXrsU0JRxXL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 44 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872699 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e85.5WyDTSgGbGTlmBxW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 52 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872707 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e7e.Z2BiTlbA03CSMFFO%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 48 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872702 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e82.qJO7zyB0I/p886lB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 51 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872706 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e7a.dzktsFAiv9z4+Z4t%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 42 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872697 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:44:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:44:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e7b.8DCQ4nbtH9AGyciZ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 43 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872697 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e8b.ZQ6Tp9Y9/P8hpAJO%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 54 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872709 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e8c.eu4OhocxRyhzZFOx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 55 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872712 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e8d.yPLCAjXTr5tS/8l6%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 56 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872714 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e8d.oyZIfV5VJa2ij2eL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 57 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872714 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e7e.zPTKYRP2HPamkATK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 47 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872701 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e91.xx4qzVp3eRIpDmUC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 62 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872720 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e94.O9a58SRMJq9wwjCB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 65 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872723 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e7c.SxH7MaASq2nYvGeo%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 45 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872699 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e81.AgtKQ0bdCkFs1vV3%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 50 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872704 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e8e.rQVxnMvOANZSHKTW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 58 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872716 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e90.1P+yobOk5gDzh9ZZ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 61 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872718 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e7f.V/M/5F8q6t6QT7l3%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 49 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872703 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e91.25647tNgto0NJ8ZR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 63 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872721 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e9a.rLjAQZy5JKDUfIcg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 72 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872730 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e93.cJm51SKNU/1HJ/Xd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 64 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872722 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e96.uMriNY7T7tGkmGhc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 67 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872726 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e99.gF05ppp/PVYK+W/w%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 71 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872729 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e96.NS2SZqm5uaRWEfvE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 68 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872726 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e8e.bWcXnUj18PrUhvxM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 59 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872716 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e95.LjfGetoI+cZJgHXy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 66 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872725 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e99.8aEmGf0dm3zQTmg6%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 70 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872728 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ea2.RdFvevVSqoxnfpm6%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 79 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872737 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e9b.DeYpdHaw8Fp3sH4I%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 73 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872731 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e9e.RYzzlwMTvehkbJZO%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 76 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872734 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e85.9WmUpca0IPUtJt/I%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 53 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872709 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e8f.8NL4Zf2Gc2U3q+Jh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 60 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872717 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e97.wESS+fvhjdhX0eye%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 69 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872727 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:37 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ea1.3BEIfir56W6bFwBt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 78 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872736 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ea3.2AKSF9C5awJlsP8r%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 80 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872739 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ea5.1J3quJk/259P2d0e%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 81 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872740 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ea5.ypscwCaBSURsRqSc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 82 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872740 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e9c.Ji6PCObnqFfUqESw%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 74 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872731 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e9d.OhxFMYxMhezzbEuN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 75 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872733 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ea7.oXWwMO7EFyleUUvo%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 84 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872743 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ea8.k2f2mDoDX/N8la4g%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 85 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872743 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65eaa.rfKGIvfbxySqTmUk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 87 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872745 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ea9.yiYAEyq3JUWvDvqn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 86 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872745 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65e9f.CIt/H6vUap+IpS6v%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 77 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872735 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ea6.E9vlkfJWdBKaIV5Z%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 83 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872742 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:37 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66045.fhdDdXU202+Iesbr%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 473 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873156 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66046.Jux7jgALmLW5ZfYy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 474 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873157 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66047.y8ytWpX8472ShI63%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 475 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873159 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66048.B8rCuJbuYMBQo+/f%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 476 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873159 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6604a.9T+r6x8ZfJ9A4tm5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 477 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873161 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6604d.rxtCf+A0ynf+q8kk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 479 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873164 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6604d.P4AAPW5eVyDVL1Eh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 481 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873164 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6604b.RjDMurm9B4mPMOWF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 478 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873163 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6604d.fvxS3Y4ppxQ4CR8I%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 480 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873162 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:47 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6604f.8r5aZWy7SykF0BGK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 483 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873166 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6604e.TMsv0zLTgj5iKLtJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 482 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873165 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66052.1yugcInmVsQWvyAT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 486 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873169 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66053.dzeg1aZOyyjLFtAh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 488 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873171 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66051.wBe88UbZO0fPlt5D%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 484 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873168 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66052.A85htmsfL0j8CmV2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 485 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873169 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66056.WElNIJr6ZNABOoso%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 490 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873172 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66053.w/crW51B5GUXYpYe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 487 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873170 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66056.FoHZF5mfhQTZLpbA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 489 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873173 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66057.xHrGAE2yDhFrG+FO%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 491 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873173 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6605a.ZAGK4zcqbbddSDZo%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 492 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873175 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6605b.hHqnx+3tI8Z5x7jI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 493 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873176 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6605d.KXoapVbEcLOS8a7A%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 494 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873176 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6605e.dstZOHa6ZuZdeRp8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 495 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873178 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66068.UOwESTkb+AeFmNM4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 496 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873178 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6606a.3OFmg58zPcEuVVA9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 497 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873193 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6606b.aTSWU762smgLIHtC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 498 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873194 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6606d.8JDpBMgSgZLQ4iDt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 499 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873195 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6606d.p4gsnDFD7ymfRsY1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 500 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873196 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6606e.DNjFvpW+CvRy3VT9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 501 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873197 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66071.AVo305AgXIfn6wf6%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 503 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873199 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66070.2VE+X5U68hv+nFJl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 502 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873198 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66072.CMdY50/DIe2F5rFd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 504 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873200 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66074.VqNa/1AGWl9/ZFjs%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 507 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873203 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66075.1HtER+ryleD7YIMb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 508 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873204 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66072.d3rtEt6KN1ywH+Iv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 505 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873201 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66073.pRDImpT7XhXBSgyQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 506 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873202 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66078.lIqiRVDvUobVvUgI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 512 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873207 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66076.wZi+g60hS/el2k1D%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 510 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873205 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66079.yvjExd3KjSW21Ynq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 513 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873208 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66076.Rj8oMuZM3lzt/XD/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 509 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873204 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66077.SBrx97nV/jviIvo4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 511 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873206 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6607b.y4wQaavf5ixxX72N%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 515 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873211 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6607d.TxPwoLx0wOoTFkXt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 517 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873212 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6607b.U0JebegnxpYrw4j9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 514 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873209 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6607e.S3HRfC4/8m1a8jQO%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 519 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873213 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6607c.VxJ8rexkLeeCFiQf%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 516 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873210 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66080.bSa01IVmk8yPoSoN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 521 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873215 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66083.02BTo5OhgkqVnAgu%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 524 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873219 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66083.YIDsMxgNkL1aCRMy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 523 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873218 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66085.0kdIOvtCBzRuP3o8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 525 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873220 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66085.L8U/UFnTNwMqXpR2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 526 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873220 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6607d.mS21MOmPzlAzqkzQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 518 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873213 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66080.WGhrg7EfwbaInGxk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 520 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873215 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66082.mzoLbpTqf2x2zXgl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 522 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873216 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66087.GAn68aI1omnK4+0m%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 527 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873221 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66087.1jM4qB81vEIluJ1r%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 528 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873222 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66088.82zUHReiXZ+9lS+P%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 529 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873223 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66089.ucxeiMMsSVsF3m39%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 530 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873225 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6608a.6pv11WYcm0hDVvWg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 531 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873225 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66090.XdgctoALR3s5s3p1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 532 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873226 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66091.B0To4Jmq+iJnXzP4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 533 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873229 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66091.PbXXahXBfkoVJywM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 534 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873232 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66093.L1iEYIgZ4nAev7xD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 535 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873234 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66094.1q0QKQz3WU/NKk52%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 536 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873235 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66095.EB31KjH0/29Ib9pK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 537 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873236 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:53:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:53:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66097.y6VQRHjX991t1XsA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 538 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873237 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66098.KT2APPd52YlakWAw%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 539 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873239 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66099.1O1IKwSrRM6+rkMI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 540 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873240 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6609a.52kSf28VPxD7ULlq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 541 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873239 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6609b.7f7bw17uFiGohTbw%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 543 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873242 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6609a.ruWOXw5WKn1MpUDs%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 542 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873241 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6609d.umwyXT40PaTbIcfV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 545 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873244 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6609c.6z3wpKusDR39BmJk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 544 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873242 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6609e.bGCJPzD1Vx3/LQDm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 547 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873245 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660a1.N96127nlBSNj4DfL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 548 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873246 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6609d.c2skL7QYlHADtbNq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 546 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873244 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660a1.xXGJeSoFPn7s+2k0%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 549 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873246 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660a2.Z8sPZrjHgxYT7Gx+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 550 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873248 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660a5.LosPmlKJvP+p3LXH%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 553 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873251 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660a7.envjwj192yNI3plg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 555 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873254 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660a3.LyM+FXFSFHjuQJb+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 551 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873249 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660a4.2rF+aFyC5i2GFNd6%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 552 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873250 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660a9.gnZeqsTqn5Q7+YvC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 557 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873257 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660ab.KE0wZDxCa9tSn6Op%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 558 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873258 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660ac.kzEYJ5GPs+2gMNaF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 559 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873259 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660a6.xVqHTkg6PCi/A7aB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 554 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873253 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660a9.mfJgdghJ27yhLVoX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 556 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873256 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660ad.Uyi1D1r5VjQB+xak%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 560 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873260 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660af.YQlgFS4uxh9rNQrl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 562 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873262 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660ae.TgGPjh1NZFR08/GJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 561 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873261 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660b0.BMzVp6+hd7HzyOnV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 563 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873262 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660b1.n4mZX1M/tV/Np9DR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 564 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873264 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660b2.m8mdnCGE3Ujhf2kV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 565 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873264 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661e6.nLXRwk5f76AjVfq9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 855 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873573 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661e6.af34r1mWk7VHaBJ3%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 856 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873561 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:49:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:49:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f98.alzT6ZwtmoLTvqF5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 311 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872983 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:49:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:49:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fa5.4wPUniXYYxGb2GVU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 323 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872996 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65faa.oSfNhYvFt5rlaIgt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 328 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873001 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fb8.YM1HVmIBnS2eYdyj%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 343 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873015 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fce.K0jXC5JXecEYbvvh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 363 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873034 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6601d.Cef+VI6V9Mn4hydO%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 436 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873116 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66041.fexdd8yuMgwD70+m%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 470 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873152 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:37 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661e9.z0tbbnQ1AO1LisxV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 858 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873574 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66015.+q0io4G27CKe+s83%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 429 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873108 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66033.w2ZHFdgozNG3UjD5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 457 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873138 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:49:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:49:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fa7.wDo2CRI5zdZgGx15%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 325 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872998 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fa9.A5XG2t7x2TjWKkYh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 327 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872998 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ff4.Hmflh11rM2lLfsq5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 399 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873075 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66021.NHWcEJB2Tz11xzUB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 441 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873119 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:49:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:49:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fa2.uE0pjE1BWGeOESYe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 320 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872994 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6603a.U24scvjgmyL6ccDT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 464 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873145 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66030.UELYkLWKccXbC4Cv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 455 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873135 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66042.DfUz2bVbFCMbJUub%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 471 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873154 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66017.zbtgou+cr8IDHFDX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 430 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873109 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66025.z52qsqz3nRVbiPS4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 445 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873124 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661e8.nz7g64DvJ59Y2g3f%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 857 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873575 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6601a.waW7uAleeITS69Yi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 432 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873113 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66027.UhD4h5mkdMjmp37B%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 447 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873125 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fcf.HFbNIPEnb6l/ebfM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 364 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873038 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fb4.sWz5pe8ZDEF2H9GD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 338 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873011 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66039.C17t/GMnu98rTR8T%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 463 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873144 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fca.lJbmMKzvhz/3AQHZ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 362 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873033 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:37 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66009.7Z2lm6rzhR/yhML2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 415 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873096 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:49:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:49:47 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f9b.QnMQRilgQPBuDrET%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 315 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872985 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fe9.zjM/JzdclXdd9Cph%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 390 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873064 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fa9.gUP5FedXUmPW4niC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 326 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872999 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fb2.tTOColBG3DxIDzz4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 335 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873009 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:47 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66013./YPaDzdl4ZTmMoJ3%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 427 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873106 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ff1.ARxp/ntyvMXi3owd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 397 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873070 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66020.4AlF6JTLHv4bSnY/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 440 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873119 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fe0.tUOO5yusj9G1LwJo%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 382 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873055 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66203.MnlVKDBOQJREPisC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 882 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873600 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66205.VdbdoNIlwh75xVEB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 884 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873602 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66021.mnxKQKC/5xOzPcGN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 442 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873120 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66036.fMb/9L9co7GRDHZZ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 459 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873140 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661fe./URP0XfMxjzGx7UU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 878 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873596 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661fb.GqhvfQFk0eDX4Nok%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 875 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873593 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66204.bdwJc6aD6uzwSeEn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 883 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873601 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fb6.lGVju3YSfSQz5lc8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 341 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873013 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ff8.tv5T7J9d/O9u0plA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 402 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873079 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6620d.FbxJ2yIJqC5+4Pb0%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 893 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873610 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ff5.+79rDowQEWe8HnKE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 400 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873076 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65feb.HcOWRXMKcr7GgIWp%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 392 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873066 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fef.98nrFC1exz7rToxS%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 395 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873070 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:49:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:49:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fa4.JMka+tN1Xp7XcwEQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 321 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872993 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fc6.oLGsXEhj8Il58o74%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 357 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873028 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6602c.+sWxHn/PuA7gi4z9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 451 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873130 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:47 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fd7.Lhkfe8M+o3FfqFl6%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 374 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873046 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fdc.ruqietjch3yc56iA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 378 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873051 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fe2.RQF08FTZMnS7LEPu%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 384 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873057 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fea.vg/pKy4io/ps6KD1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 391 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873065 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fc2.1SJ5HVvB/kR437nP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 353 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873025 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fb3.kfeyhHkdFMnc+IMr%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 336 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873008 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fdc.8eCzDkPGdnpybiUQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 379 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873049 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6620c.zKpGLlji7FSROU7z%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 892 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873610 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66008.IGafGe1AqBsMx4bQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 414 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873095 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66202.YLROGTcDx/q7DOgf%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 880 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873599 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6620f.Rluxkc9yKSjV3EEt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 895 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873613 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ff1.CT3YRQ+KDq5zyBb8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 396 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873072 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:54:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:54:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf660b9.mWjY/aIA2NhDb24/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 569 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873269 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66208.zu3RM3yysWJ9cCba%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 888 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873605 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fdd.J6+efTFa37L+NxOv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 380 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873052 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66209.sAyNMIhn6c+4kFPq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 889 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873607 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fbb.n/YPZMBVXnaA6Nxo%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 346 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873018 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661fe.3ErRUYn35yCwgShv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 879 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873596 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661fa.dfgKJ5eruXjMsaW7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 873 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873591 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66205.arXDqWpa48IzZ57G%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 885 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873603 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661f0.t4QW2z/qTCK2vCM2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 865 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873583 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66211.prrS6f342/i+B1Mb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 896 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873616 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fae.l1i14btt9wYHT7f9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 331 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873005 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6601b.TzyKS09RN45Dc1cV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 434 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873114 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fd6.IispgGLqH3vhaE0l%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 373 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873044 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:49:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:49:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fa0.Gzp/qmK/P1rnC11C%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 318 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872991 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6620b.vDAupSxLecyTqF4Q%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 891 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873609 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:52:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:52:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66040.E7txBkuzTmf7cRwS%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 468 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873151 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fe3.aobL86iBBFJWQbGS%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 385 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873058 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6620e.tkV8IE9hac9rry4p%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 894 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873612 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66202.pTnxQ7c4u5H1IPDm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 881 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873598 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6600d.EiMPWfHgwoLPYZjU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 420 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873100 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fee.p1nsLNFR88o/UcnJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 394 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873069 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fd1.Mni5A69OzKRVqpPh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 367 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873039 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66209.kyzF2fypSSsUW6QZ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 890 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873608 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66207.z+69aIA6zpwyX0Rb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 887 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873606 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:50:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:50:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65fb7.pZOfkLFWsVX0fGRN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 342 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873014 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66206.c6UNARm91H5E9Tfs%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 886 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873603 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669e2.a+ddgl9bgXlv2HM4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2865 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875608 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:37 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669e1.zth7JWUW1kQjx83T%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2864 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875616 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669e6.pWDZhjF39wUazXBB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2869 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875621 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669e3.ChCiQyRe0CuKbu+u%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2867 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875617 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669e2.1nPw8vm4YOoytsmG%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2866 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875617 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669e7.Gqlb0d8a9TJ+lVKL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2871 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875623 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669e4.Qm8lkZG28bf/pnq1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2868 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875618 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669e6.awmauZTmObc0uLsZ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2870 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875619 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669e8.ZDj53LUvExT4vMnD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2872 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875624 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669ea.gxoVOnQ5ohNUHtQ9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2873 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875625 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669ec.WUv/Tjps1k9GyXko%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2874 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875627 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669ed.AkuPGSXPj2JBUevp%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2875 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875626 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669ee.ZL2Qh56xZSqQ53DG%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2877 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875628 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669f0.C87wEqSoLeOcDA5q%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2880 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875631 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669f2.GyvYE+rIj5LxYCr5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2882 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875633 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669ef.ULhY6fEY27jXhLDg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2878 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875630 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669f0.H141dOBgGbFnoSLF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2879 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875620 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669f0.dZ74oewcAlo1TG2Q%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2881 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875631 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669f2.hZ+1K8qajbEie2p8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2883 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875632 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669f4.lhFCQRt7ut3s0M51%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2884 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875634 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669ee.RgooCjeTI7S5zYlz%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2876 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875629 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669f6.Wty1Wu7TpODy/DGL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2886 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875637 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:34:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:34:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669f9.tJ9tsJBtHpYZFM0o%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2890 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875640 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:34:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:34:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669f8.TQWV6yhQ2750PURz%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2889 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875639 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:34:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:34:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669fa.OYUQ1kDi22wF3Twc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2891 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875641 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:34:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:34:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669fc.vNMDkNRa6QK97DmZ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2892 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875643 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:34:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:34:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669fe.f8Ux4nLT3Gy2sSCR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2894 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875642 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:34:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:34:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669fd.3flzy6uR4A9Qz357%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2893 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875644 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:33:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:33:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669f7.NZn5TXJEwLuSiDYB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2888 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875638 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b36.e3MaTbbMZekfpNMj%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3195 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875956 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b35.ng/QPljna32Hdfi1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3194 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875955 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b36.mOxrGcsSJ058omYf%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3196 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875957 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b38.qbJQQV4+NUZVDzAl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3197 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875958 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b39.sWWv9RefQqLZtRQf%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3199 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875960 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b3b.9ywWIpMmwzJMWgR8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3201 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875962 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b39.qG9LLQaIs77nvYmN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3198 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875959 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b3a.8jsSuHhbZfYNmHf2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3200 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875961 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b3c.9jtC3eBCnX4Wvjzv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3202 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875962 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b3d.ZHgl4obPqiIfV7nu%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3203 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875964 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b3e.tXdjab2G+svYT6BY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3204 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875965 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b3f.2zmvYej56zSCTg0s%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3205 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875966 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b40.2O1h/b9XIx/PSzPi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3206 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875968 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b43.DswVDOwTBlQHc9I+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3207 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875969 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b44.OZgAfaLBx9GU3eun%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3209 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875971 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b43.xi4uJpUhKV0Bfp+P%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3208 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875970 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b45.VX0SJyiOtzJMRPK/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3210 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875973 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:01:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:01:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6624a.ukzFOyh/fh51jxZ1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 950 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873671 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b47.W+37CbZAOIwxi0Tg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3211 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875974 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:39:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:39:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b47.IuMGU6kSqLMO9dQQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3212 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875973 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:01:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:01:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66241.5kcGWgG9FDy42/Tv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 942 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873664 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:01:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:01:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66274.Zsi2kMGow+gxksHR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 989 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873714 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:02:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:02:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6628f.Xe1IdE0Ap7rJinw1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1015 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873741 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:03:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:03:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662d2.PkiFB8zBSAaRZ6rd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1075 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873809 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:01:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:01:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66266.Lp2EpE338JzVbRUk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 976 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873701 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:02:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:02:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6629a.AD1GpKnSk6Ov7N4R%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1026 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873753 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:02:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:02:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662ab.0rehHqttvaBO1GcN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1040 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873769 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:24:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:24:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf667a1.n5nANOkI/Vk7wwlw%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2302 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875040 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:03:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:03:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662d6./3a6CWwRsVoDU+TB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1078 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873813 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:31:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:31:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66968.3LSMCWYNcDkKxb4/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2744 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875495 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:24:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:24:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf667bc.Dj+fcE9pwf55tzzy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2326 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875066 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:24:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:24:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf667a0.mznt85JfzN3Mstrd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2300 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875039 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:21:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:21:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66712.LGarUN1z5ew2x9mn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2164 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874897 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:23:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:23:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66779.cvBku2Wwq6hwx8HV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2265 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874999 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661fd.yJKFKjBJvPLH5/ud%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 877 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873595 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:21:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:21:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666ec.WeZ0fPdg/kl+zjP/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2124 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874858 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:03:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:03:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662eb.E23Dhgl5LvspokBp%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1098 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873834 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:02:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:02:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6629f.Yr4TbTMBOv+VOoBY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1030 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873754 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:02:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:02:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662a5.Fhfmie49RS8UP0fu%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1035 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873763 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:22:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:22:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66753.8B/+gIrnm9/O4PJz%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2228 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874961 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:32:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:32:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669a1.t+afEfUWAgqXcNiD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2802 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875552 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:31:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:31:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66979.lQ5MpjI8gSVZeTWA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2760 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875512 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666e0.Kp2b+crfKPw5XsAh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2111 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874846 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:32:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:32:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6698c.XqK53vxCeJ83n3my%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2780 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875530 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:30:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:30:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66942.6tYkz0HejhASEnEb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2709 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875457 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:02:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:02:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662a4.9wQ1fSmWU62Dpmvt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1034 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873762 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:03:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:03:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662bc.eSZxxN3wNtqdXbZD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1055 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873786 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:30:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:30:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66921.ven+eA6eJmR5QdKI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2674 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875423 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:21:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:21:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666fb.ABtVdotedjChmLDN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2141 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874873 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:01:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:01:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66254.12iG0/ndQM1j+GB9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 960 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873682 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:30:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:30:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66915.8+m08tvPC8xlPa6Y%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2662 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875412 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:19:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:19:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666a4.gkdjy0dfU+S638j7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2049 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874785 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:03:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:03:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662b4.5eZYp5AJYNU2kyvU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1048 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873778 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666e9.xQTjoSXV4lKmtnbW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2120 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874856 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:01:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:01:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66250.fnaeHhIpZnpXHXo6%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 956 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873679 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666b6.6dVMvJQHs88EGioe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2067 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874805 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:24:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:24:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf667b8.f75Rp1hi+fx8rT/z%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2322 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875063 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:22:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:22:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6673d.zb0Ozprv1ilvloaU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2204 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874939 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:23:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:23:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66788.0CaIRHSPwix762mb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2280 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875015 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:19:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:19:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666a2.SlvCZLN5p+Zoc0JQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2046 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874784 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:01:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:01:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66252.Cvj9SCozxf9WRC8q%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 957 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873681 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:32:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:32:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6699b.bGnBeiEFduURj0/H%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2798 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875546 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:21:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:21:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66710.jS9PaCMoDd1/1I3U%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2163 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874896 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:01:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:01:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66243.NXJvoPv53EmEKZs+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 943 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873665 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:34:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:34:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66a0e.9BLhMIM3vkxfjBYj%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2909 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875661 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:32:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:32:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66999.E6HAEdCu0iiy8cy3%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2796 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875543 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:30:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:30:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66922.j3Y6TvspEfSe2MHa%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2676 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875424 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:29:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:29:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf668ff.M0000rM5+rspz3H1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2642 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875389 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666e4.Aa8jZhDRWnqg+3xk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2115 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874851 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:23:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:23:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66784.9PH52IAwKBe+3ot5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2276 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875010 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:21:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:21:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666f3.sFGrrC8BcTTWsV7d%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2132 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874865 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:02:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:02:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6627a.DG5o00IQamSXUnsq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 995 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873720 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661ee.10G3BHdYCDckk3ez%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 863 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873581 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:32:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:32:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6698c.CskZnWWeXZrmzMod%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2781 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875531 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:23:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:23:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6679c.4jk5ruwP0WaQqaz3%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2297 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875035 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666b4.jZRo71EzlZnqXGzE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2065 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874802 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:34:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:34:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66a0a.zLO8kfVqh1/7Y0im%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2906 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875656 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:24:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:24:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf667ba.tEiQdddyMeOg2sVn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2324 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875065 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:22:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:22:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6672b.rTTm/RhKKH/bjmub%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2186 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874918 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:31:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:31:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66951.WdPP8v6yhrbuiRMV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2724 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875473 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:34:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:34:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66a08.aAGAWlCDX72ZlguP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2903 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875654 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:32:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:32:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66990.bDyUuaOhvpJsdh85%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2784 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875534 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:30:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:30:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66940.k+X+q+v0Mc7cPZp7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2706 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875455 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:22:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:22:47 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66757.slUdfH4L0YdK0GKj%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2231 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874965 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:32:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:32:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66995.v3mrpEqz1itUIn+v%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2792 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875540 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:32:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:32:37 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669a5.joYe+S60eKm8zMl4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2807 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875556 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:30:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:30:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6691e.XbzD73aJIluS8n1i%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2673 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875419 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:22:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:22:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66728.eZyQjyhaDQaNCMUM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2184 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874919 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:01:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:01:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6624c.MhvQpof1l3xKNIuP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 952 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873675 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:22:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:22:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6672f.Mq+MSRlaDjLQy15b%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2190 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874926 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:32:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:32:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf669a2.bcpeEKuP+4cBap02%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2804 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875553 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:09:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:09:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66443.vZkr3u7q6zZ/xnM2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1436 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874178 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:03:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:03:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662dd.nQogaL5OA/KVJh9K%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1085 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873819 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:22:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:22:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6674f.penHw/6DL81Z8tNF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2225 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874957 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:23:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:23:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66765.7NeeVzuwRYJbC9aN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2246 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874980 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:04:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:04:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662fb.69vo70Bfjy9KIWkr%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1113 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873851 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666e1.2vcz1tsymCPOnZ1S%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2112 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874848 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:04:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:04:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662fe.2RFqPxJxWvwB6vfJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1114 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873853 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:02:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:02:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66299.bwXxSpJtghRLW5nB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1025 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873752 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:01:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:01:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6625a.LPL/8EtOpfc63X9U%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 964 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873689 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666b7.gkJcdCsPpzbOnqB5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2068 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874805 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:32:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:32:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66991.8seqc/J7AjIdXleo%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2785 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875535 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:29:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:29:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66907.2+g5JP0fmDbjgPB7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2651 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875398 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:03:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:03:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662c1.+d7UC+zgQclRoFd8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1060 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873790 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:02:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:02:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66284.36mNvSEvVRmHdlRI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1005 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873728 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:24:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:24:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf667b4.eHJd1s962V/o2gHW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2317 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875055 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:21:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:21:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666f5.gwlIo4cqkGGx5tGJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2135 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874867 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:02:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:02:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66299.y7JMcgAtTlHS8GkH%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1024 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873745 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:01:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:01:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6623e.zoYaGZVTlvbF/DcF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 940 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873661 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:23:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:23:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66776.eDbFcBh/gef+RY0P%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2262 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874996 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:31:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:31:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6697e.1x53E/25vXFg9GPd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2764 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875518 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67449.9OPFcWmkYXEBSFuQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5519 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878280 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6744a.m7YX+V3tSec90hqU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5520 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878281 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6744b.rlrLSg6mAnkIQ2Oo%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5521 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878282 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6744c.r+Y+ZtgkFtXJ9V5X%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5522 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878283 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6744d.rRXRSprZrHSqETNA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5523 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878284 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6744e.QkAPtBswQSodfD5V%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5524 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878284 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67450.cECVpzCEyGCR3lWG%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5527 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878287 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6744e.fU/PuRNyTOmlVSiQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5525 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878285 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67454.D9IG6XAmKH/AkC4+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5531 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878291 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67451.nEYsN4YsYNYRJTnU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5528 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878288 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67453.6qENLd7ct+3s1slm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5530 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878290 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67455.N+Q8CEilUjYt3vhb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5532 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878293 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67457.w5qR6xOsrYsupZgW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5533 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878294 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6744f.6XRUUUP+P7yNxEKh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5526 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878286 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67452.PBmC9hNDH4rnbuiM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5529 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878289 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67458.0Rmp7wG0vleRXR2m%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5534 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878295 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6745b.epH74Rn4ZUePWsx1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5539 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878298 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67459.1Q8YeaQIj6qwqtrC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5535 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878280 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6745d.RkK3Ewm5IorKZHgt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5540 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878300 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67459.kpB3H3XT1Z8Bs9iS%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5536 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878296 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6745e.vzjxnIwm0lurRcp7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5541 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878299 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6745a.d/q+vTFtS1ykUFCL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5537 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878296 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6745b.FC9Ns0NITu//Ty5e%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5538 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878298 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6745f.maFEqI8ElT9v5cgd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5542 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878301 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67460.OwVu75mN93zGM7YX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5543 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878303 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67462.oXMnN1OQRpnwyhmt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5544 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878304 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67466.quzh0LWxJOh8sdSR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5548 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878309 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67464.qSgj74xxAhCdjqGl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5546 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878305 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67467.z3yE4/H+HsdQA4m5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5549 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878301 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67467.v6ucZJTpE5qXFsDC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5550 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878310 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67465.KlYu92bAUA0nSiCR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5547 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878307 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6746a.qVvw1E7dhgOYaqNn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5552 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878313 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6746b.R26TgkLA2GkP+uGa%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5553 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878314 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6746c.Pe4tjop4QfPUwF/8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5554 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878315 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6746f.chhzc8aXe2QTIJ0e%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5556 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878310 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67463.2DvuL3J/YbPa+K27%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5545 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878306 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67469.4hZmJ2We7UpipchX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5551 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878312 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6746e.JDIyTqIbFY1r3rtm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5555 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878317 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6746f.I5JkGS4zkaEVo2xE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5557 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878318 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67471.Z1ywxzHd4gEvmekN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5558 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878320 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67474.E1VadYR13KKAzIAg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5560 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878323 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67475.ne90uS9PJaZR4nM5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5562 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878324 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67472.FUa7RuW1baKlug8b%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5559 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878319 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67474.fqX/S8nBOXgmpeQk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5561 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878321 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67478.WkUmS2DpKcJjsJxx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5564 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878326 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67476.xNwsrRbovOF0rhGc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5563 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878325 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6747a.cjjo5yxyH2vZLPNT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5565 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878329 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6747b.16F6Zu8qdkrhJehP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5566 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878330 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6747d.hHM84mH71ld4IuRr%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5569 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878332 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6747e.etU3iRLsrtKP2Ln0%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5570 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878333 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6747c.owN0qVzqZE/HgO1p%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5567 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878328 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6747d.qn11GGmTuS49rFW4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5568 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878331 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67482.1bNMuZ6g4bjIsp4b%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5572 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878337 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67483.Zy4RYgAbWWJWdxOq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5573 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878338 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67486.GetKTow4FjjLk5a9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5576 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878335 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67486.Fx5Mv1OGKF99V3Up%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5575 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878340 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:18:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:18:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67480.YA+FyTbFg9FtKD1q%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5571 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878334 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67485.VIDBviLQE1eH684A%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5574 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878339 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67489.u+6ITSKkBegyMONz%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5580 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878343 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67487.gGVcQq4gvDcezPJW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5577 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878321 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6748e.Mf10+d0cIu4BEeIi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5586 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878350 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67488.exPKkCxBm2Pxfu8n%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5579 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878342 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67489.OgsA3g7GzkjfgM6l%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5581 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878344 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67487.MbZRM1rQ9zTX/evq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5578 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878341 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6748a.mbsAHwIiZ2J15GKo%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5582 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878344 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6748c.BywgefD0zILo/swa%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5585 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878347 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67490.lJKif6YMq8AnoOAh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5588 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878351 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6748f.IqtupRJWjaN2CGvB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5587 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878348 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6748b.mNy7cNnjFCS19DPi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5584 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878346 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6748b.GGETST1rRAO5hWxv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5583 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878346 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67492.Zf6b+pxcqxOdC7/w%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5590 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878348 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67495.WQKkzSnXYoONvSbz%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5594 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878355 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67495.OiqOfuEMlU27YKpB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5593 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878354 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67499.qWGZfntztHG34nKU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5597 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878360 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67492.1S/dI0OY4Z6y08da%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5589 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878352 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67494.SIUB1PSa8swZEPSt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5592 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878353 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67494.V35WFTAPYHvDHVdy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5591 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878355 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67497.5HoRr92hRf8bq+U+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5596 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878358 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67496.028ba+zLYXuYXH5g%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5595 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878356 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf674a0.NfvKlVHLIYGrNOkC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5603 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878365 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6749b.56qKVlW6s+0kjYD1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5599 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878359 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf674a0.BcaHwBBps+5HU+kA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5604 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878365 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6749d.3RwdRx/1oAM0cyWY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5601 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878364 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6749c.2GzpiZeh+j3pZHi1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5600 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878362 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6749a.PtUCHR3PjhuX/vYK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5598 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878361 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf674a3.Ds+lxMoVxu+hwLIY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5608 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878370 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6749f.GdlJJOxqgO09diIX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5602 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878366 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf674a6.b1WkLLrEl779wTnx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5610 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878373 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf674a0.9lSEyhmyurPHOYjg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5605 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878367 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf674a2.1UkYE4YfPUZYfD3R%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5607 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878368 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf674a2.MH7bQjDSYMzNhtgN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5606 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878369 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf674a4.peYswn10O8eH0ITn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5609 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878371 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:02:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:02:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf662a8.TEjwYZMmvzWLpU9Z%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1038 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873766 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66226.7qegLfoIzhDTlVx9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 919 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873636 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:04:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:04:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6630d.etRrlthKMo85EJg1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1128 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873867 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:04:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:04:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66303.YNQ0Op8mkctgikpm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1120 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873858 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:59:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:59:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf661fc./hvt4eJybU5KtaXh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 876 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873594 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:41:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:41:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66bca.o7iI96c8XnUa4V2+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3341 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876104 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:42:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:42:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66c03./f1qwgh0J9s3DV6Y%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3397 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876161 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:46:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:46:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66ceb.fiL7cyWbyA+8zGb/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3631 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876394 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:48:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:48:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d4e.p9G8f9Wzhd5EQgIt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3727 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876492 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:48:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:48:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d71.ik6ket3w0yQHmvUT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3758 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876528 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:07:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:07:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf671dc.cf+EB9gfKGASy0TJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4894 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877658 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:06:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:06:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67195.2vAdfZHcplOJjKV7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4821 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877589 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:15:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:15:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6739c.N+Sue+A4PrwsU0bH%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5349 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878106 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:15:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:15:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf673bd.QdpfcYr+5C+Ed2P2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5379 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878139 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:15:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:15:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf673b2.qAcnIoBpu7/YdPwk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5368 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878125 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:08:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:08:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66406.UBdCbhXuGR6yl+5A%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1377 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874115 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:15:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:15:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6739f.j/CZiTLhMLczZb8p%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5352 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878110 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:04:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:04:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6712c.7VkIi0wutqczSLkI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4714 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877481 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:16:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:16:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf673da.wvWXjgn5OB5zDLIQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5409 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878155 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:17:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:17:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67424.T5+Y6HrTXunS5Ndi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5481 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878242 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:17:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:17:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67438.COE+/glwWBdzG51E%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5501 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878262 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:06:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:06:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66379.QtE1fdzCC+nokQu5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1236 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873975 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:08:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:08:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf671f0.7oKKTXfm3+8UdURf%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4914 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877679 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:48:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:48:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d68.vlm5+Dxom8J2/qXG%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3751 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876518 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:05:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:05:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66346.t1QZbGXJCtLlRQc4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1185 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873924 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:05:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:05:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67168.qRofnp7Dfnlsa1CF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4777 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877542 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:16:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:16:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf673e5.meGZrwM/PEr/MKj+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5420 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878179 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:17:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:17:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67430.Ty5oqPu723aQK8kt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5494 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878254 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:08:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:08:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66416.+Kdv6Eute2QnlNc8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1391 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874133 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:49:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:49:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d7d.uodn01tWssRGIcxQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3770 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876538 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:05:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:05:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67155.46qjLFam862ZNqZb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4757 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877522 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:06:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:06:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6636f.0dmii3f16OxYdw87%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1225 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873966 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:05:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:05:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66353.o0VMI6rvD3rl+0ac%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1199 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873938 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:17:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:17:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6743e.pkm/jH+HuhwuuImy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5509 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878269 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:09:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:09:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66440.JfixrRVUba7dycYb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1430 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874174 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:08:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:08:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf663e4.134wxKMJRObpTdi4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1343 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874083 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:08:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:08:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67224.+GeDNAG75Dye3uLQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4967 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877730 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:06:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:06:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf671a8.rURJVGmuFYEH3Evf%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4842 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877607 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:49:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:49:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d96.rnj7OndkJ/8SwOL6%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3795 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876543 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:08:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:08:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf663f2.CjEW8QwAi1xHMd5i%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1358 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874097 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:49:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:49:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d9a.+XEzqWY4cgwUIEZu%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3802 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876569 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:04:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:04:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66326.zY30/bsLtEP2NtoC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1153 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873894 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:17:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:17:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67414.4NS3gmoJ0Z4ABmCT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5464 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878225 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:46:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:46:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66ccf.g6n8H7C20+PcsTrQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3604 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876365 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:05:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:05:47 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6716b.p8kB1jPPRU2PznzM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4782 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877545 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:46:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:46:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66ce8.8TseBxKojlWFlqfa%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3628 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876390 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:46:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:46:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66cd0.kiGWs7BVGzgi+XKP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3605 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876366 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:47:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:47:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d2c.OW3ddahBlLOd6Hs1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3693 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876459 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:07:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:07:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf663cb.cUQkojNmlBoxTOoZ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1316 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874058 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:49:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:49:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d8e.bYsN1Q23Os69h9rA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3787 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876557 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:06:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:06:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf671b0.AQYG0TblmhDrGLTl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4851 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877615 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:07:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:07:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf663c5.LWBkVDnek9bE/zcJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1312 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874052 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:14:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:14:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67382.LHwhmeNcpjTBhdZk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5323 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878080 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:08:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:08:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6720d.zeIu9trTgqX77Z1p%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4942 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877708 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:08:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:08:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66417.8LyBHv/MT1iclPpz%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1392 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874134 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:07:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:07:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf671cc.D4rEA5XUgAgsGDvd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4876 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877642 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:49:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:49:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d91.RyuEyV+ZJ/XR1sGB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3790 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876559 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:05:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:05:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67170.ik/dVjd7/da+MNKA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4786 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877551 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:45:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:45:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66c94.qyJRkqbJSt2+VlmB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3546 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876306 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:06:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:06:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67196.Yv9wQI6Zq4VaNZZz%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4822 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877587 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:15:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:15:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf673c7.fGjqH7uF74k3PZKi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5391 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878150 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:05:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:05:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66363.9Tjk+Ul/CwLb49GW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1214 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873954 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:06:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:06:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67179.w/rTEaZ/hLHh8kbn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4795 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877560 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:08:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:08:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6640a.fr7CeAWNc8U2dEH5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1381 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874121 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:48:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:48:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d76.vTLAK/2Vev7BrenS%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3764 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876532 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:16:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:16:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf673e9.O14T+nJh9tIUYKcu%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5423 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878183 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:46:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:46:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66cf4.W2EZ0JaXSG6LKXYN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3640 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876403 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:06:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:06:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67190.rX2AGq6w117CzF/2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4817 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877582 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:17:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:17:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6740c.l8JpYgvbsYfJDn+r%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5457 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878219 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:07:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:07:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf671de.UiYzo2yBt92VXm40%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4897 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877661 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:48:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:48:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d55.u4ZKh+hfI9ccq+YQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3734 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876500 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:47:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:47:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d1f.2OJPBgaH5DciLEjc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3682 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876447 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:00:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:00:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66218.j8DO+04MhCWQBiO2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 903 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873622 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:17:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:17:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67419.mxQMK+7GNs/K6Ypf%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5472 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878232 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:07:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:07:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf663c0.ar5JIG6ERCdC4k3e%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 1306 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874045 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:47:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:47:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d0d.x5KqWsD3r5P2e5Qz%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3662 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876427 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:46:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:46:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66ccd.ZrqaZPiPfBPyuQJD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3602 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876363 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:48:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:48:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66d59.fWeasyJ3vDpGYvyi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3737 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876503 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:44:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:44:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67a8a.AECEIyuaXabBnAQ/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7104 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879880 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:16:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:16:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf673ef.0Eaj3IS7SWMZ3Fyr%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5429 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878187 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:16:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:16:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf673de.TgJCU4MTBqmysSFL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5413 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878170 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:47:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:47:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67b49.HE40LSVGpTJ2RkVk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7294 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880072 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:16:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:16:47 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf673ff.PvMgz0RnWs28GeJ/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5444 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878195 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:35:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:35:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67874.r9H9GQEkYQkuF7To%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6563 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879347 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:48:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:48:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67b50.Wk5kyDPOJcec36br%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7300 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880078 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:48:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:48:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67b54.yS9b61wpEXHtmlK9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7304 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880084 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:17:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:17:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67416.pI5lgv/uUVMWaXgl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5465 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878228 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:45:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:45:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67aa7.Ua456pCJIWW9me0R%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7133 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879910 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:45:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:45:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ab8.rd94BFmDz8mGe2A/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7150 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879927 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:16:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:16:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf673fd.k5TwRVO2xTqeMpbF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5441 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878196 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:39:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:39:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6793e.9qtgl8BLOSAyW1eF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6762 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879549 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:16:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:16:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6740a.5YkvG4anjfoucsvJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5455 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878216 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:37:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:37:37 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf678e1.nDTtmVFeUHLLfvPp%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6667 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879456 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:45:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:45:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ac6.Jgl+hmQQcRodsNbN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7161 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879939 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:45:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:45:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ad3.AWIha5Sdv/X9RBOO%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7173 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879955 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:45:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:45:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ace.XT7AacFLc09+CV0x%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7170 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879949 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:47:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:47:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67b14.PaDjEWuGAclIi7rz%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7236 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880017 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:47:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:47:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67b28.DZOdHa1iDGYCloeU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7257 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880038 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:38:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:38:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67900.xL5dGiayFPjGBAi7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6698 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879486 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:38:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:38:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67904.HkGECwtV4bXiAyce%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6701 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879488 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:35:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:35:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66a45.RWriu8DUtiOvrreP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2959 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875716 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:48:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:48:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67b5d.oFZhpjuQCTO0bOBE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7313 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880091 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:17:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:17:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67447.BYTQu2njV5IPbtTI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5517 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878279 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:39:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:39:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6793e.gdC72N8asGH2+5j5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6760 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879541 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:38:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:38:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66b15.mRsY7KeMKfgtiC4f%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 3163 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875925 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cfa.3qa+ESh5zeAl2BsM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7721 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880501 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cf8.tM14xfunBUZPXDid%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7719 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880503 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cff.wckOEIE8V/9ngxJx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7728 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880510 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d04.UVgSQlqS/9aup1BW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7732 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880514 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:54:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:54:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cf0.X4Le2rLQxWP8733E%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7714 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880494 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cfd.MdrY+vOF/NE0pX7T%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7726 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880508 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cf7.ZcEcyUjR3xa9HBvU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7718 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880498 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cff.X6QGbvCyzo7G1SZp%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7727 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880510 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:54:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:54:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ce3.pRUIbJAa8zxfjL6v%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7700 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880482 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:54:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:54:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cec.0LcdlwWT81udrnDD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7709 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880490 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d05.eJ9puQER1ZKN6d+T%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7733 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880516 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:54:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:54:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ce9.+KpxDDHfgNADVuQJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7707 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880487 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:54:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:54:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cee.A9VKpSLmpU25VrRR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7710 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880493 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cf9.nMzffSzJEW7dfyWe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7720 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880505 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cfb.TkHRsw86P53Sl2Jr%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7722 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880505 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:54:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:54:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cf2.4LQS/dupgHoYZwTk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7715 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880497 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cfc.bBeeZG/9vvJTOo2C%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7724 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880507 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d02.glLEKzzmqidJcprA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7731 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880513 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cfb.Ns8Aqwj/0rdWhl86%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7723 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880506 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d01.Y/iXOftDQtRG+BDC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7730 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880512 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cfd.4lYmQ+BiTK6dYXgA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7725 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880508 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:54:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:54:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cf2.eUGEkpMgNRrInSE+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7716 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880496 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:54:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:54:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cf0.nm7kwj/hNhxKvkfX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7713 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880494 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67cf5.uIqNi5vcyWGRBoUc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7717 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880499 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d00.gUb89vYwG3DSQYwA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7729 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880511 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e4e.QVtTgusotBV7zpUK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8051 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880845 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e50.Cd1+O54lg+tZ5IJW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8052 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880847 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e51.cHFenrh0DDb/rl09%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8053 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880848 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e54.aWkxFXk9eDNYxdRE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8054 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880849 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e55.DYocV/iopBdIylT7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8057 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880852 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e54.4p+FNX4xf0jOIFtA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8055 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880851 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e55.7L8ygkbyIScm4+8M%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8056 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880851 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e57.vV5RL4/z69xD+D9h%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8058 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880854 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e58.aulR0v4nKArTOeWM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8059 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880856 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e59.1b4Vgd1awS+q4w2S%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8060 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880855 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e5d.sqUtddliInHS2raB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8063 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880858 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e5a.2+ZfdbeFuuLWrAJn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8061 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880856 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e5d.WmyGh7eItziAX0jj%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8064 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880860 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e5f.ZO/AD7OA6PuqZ8Z7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8066 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880861 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:00:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:00:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e5b./pveUpY1tls6KWZY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8062 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880857 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e60.5R3kzyBRKYAJ/3yT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8067 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880863 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e5e.jk9bgUUtuugBhxau%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8065 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880860 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e64.vXfyuFaNfxa0FyRT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8070 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880867 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e63.Xi/mkpHtcIphtF+7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8069 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880865 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e61.wDh3G3Lrny77jqp2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8068 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880864 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e64.Ksfmqm38zvNOVgde%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8071 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880867 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e66.q5n1ww2B6BrF5WCP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8073 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880869 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e65.M0LHEnvtFR+78jVK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8072 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880868 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e67.2BozBLMCr82ELaN9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8074 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880870 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e6a.U7CCecmmZIR1WOcw%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8076 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880873 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e69.bZQY0rgfXkZMHtOe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8075 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880872 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e6c.5JD0SByZx7m01PjJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8078 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880875 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e6f.d71/rtFsWMPjU/BK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8080 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880878 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e70.3dp2Bey70iCq4lNh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8081 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880879 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e6d.yBvi3lexj53YuzVR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8079 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880876 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e70.GhWppt1AdpC4NaAl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8082 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880879 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e73.tlsM0LGeSLwYpSBQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8085 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880881 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e74.unANvDJsShvUvGNK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8086 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880883 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e73.WG9JroUGQJXZFNZ8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8084 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880859 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e78.b1OuLc/rso/gMPJI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8090 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880887 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e6b.c8h3wRTzUAI1U0AF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8077 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880874 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e79.kEcGjt678zo8lmvY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8091 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880888 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e72.5U0iOKyh6s7PWNwB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8083 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880880 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e75.504nVPCNUDDVIPRP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8087 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880884 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e76.9Ol9bogxxS0LBDJ9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8089 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880886 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e76.6eCI44mdXVgMueal%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8088 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880885 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e7a.2iWW6XhbiPmYHT7j%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8093 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880889 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e83.jf1Buv0a51ft/SmX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8101 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880896 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e7e.HbCm1n/TiD3CyZss%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8097 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880892 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e85.CVQ3AlriaXJD8Qy4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8103 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880900 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e79.MS02BxEzQgorVoqs%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8092 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880888 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e7b.afKfpMEi6sgEe6dU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8094 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880890 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e83.xvkU+jjGwKJRV22R%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8100 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880898 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e86.9WdovMjB7Oi2vMlE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8105 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880901 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e7c.MZ8fxwr2BlOaoJd4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8095 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880891 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e7d.b3PUP0LxzNs+/t8a%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8096 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880892 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e86.uLx6TTTzONg27W+S%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8104 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880895 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e87.bN06C3Zz8bNwoLxt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8106 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880901 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e89.q47OL1WZjA0Q0qfT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8107 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880903 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e8a.e5YkEq6XoEf2fPwL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8109 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880903 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e7f.2Uqed4qpwhSrSSMc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8098 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880894 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e89.xvE1caTGlCSKTQ8q%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8108 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880882 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e80.RDgcxs8ifYr1yX0Q%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8099 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880894 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e84.lTJGMDXg1QaKMMpF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8102 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880899 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e8e.ATYZaX0+HajQurEy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8114 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880904 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e8f.O7fB3233C/+EGPR/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8115 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880909 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:47 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e8b.hWcdvFPGJmeQ3LQo%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8110 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880907 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e8d.oPNCNkgo0BclKrLX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8111 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880905 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e93.zIuEYX2NKHmE5Jx5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8119 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880913 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e94.3b1YhcnwA/GAQpcG%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8120 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880914 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e95.BJEIFsxfve45J64R%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8121 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880916 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e91.vP86EGRZIDKpt5b+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8117 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880910 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e8e.T6oTKgGogN3XOhqM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8113 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880908 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e9b.PPaWV062M0stKcvT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8126 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880915 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e91.5KxdQyCmI95bwC7l%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8116 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880912 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e8d.bPnxXNI/MoAQUMJA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8112 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880908 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e9d.n9yfJDlSlmLp6fo7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8128 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880922 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e9d.9bu91ni+Zq4D7fwb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8129 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880924 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e92.Hs72SJ2KaIxNMA7f%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8118 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880913 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e97.ptwqSVKSqFPhheFd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8123 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880918 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e99.d56MeGMXbeKRxboi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8124 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880919 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e9a.BMpLDgR4x2dR16Uk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8125 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880921 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:01:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:01:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e96.T65sKXPuXJ/57jgV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8122 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880917 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ea1.eCsiKldamUrKrwSq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8132 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880928 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ea4.aWprnBTb0A4jQbeg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8135 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880927 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e9c.o8Zg4JejmgJg5yO7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8127 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880923 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e9f.WNJm/BZlyXxykg6q%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8130 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880926 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ea6.K6RhjYMpmTe8LJ8n%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8136 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880933 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e9f.LEFbCoFXMBBO9BbV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8131 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880922 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ea3.R4ZgmjHFaE2HyMfJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8133 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880929 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ea7.ZEIo09L62nKYPJyH%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8137 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880932 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ea8.CQKsvOC9nNcaBRNk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8138 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880934 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:02:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:02:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ea4.QmlYKmu9xwR3QQGn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8134 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880931 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fe6.yXMKlJtZMaMEKOGv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8452 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881251 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:37 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fe9.DF6jGrGhtSubiYkH%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8455 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881253 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fe8.xuHhp1mhupOpfIfl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8454 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881254 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fec.x/ChBgdbHESzSw82%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8458 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881258 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67feb.Jf9e4T55X/7upCsk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8456 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881257 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fed.lLiyU/nDVLvIymo1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8459 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881259 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fee.Z4zf5RylNeMWeJcJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8460 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881258 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fe8.idU0sbcv2QVeZWze%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8453 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881253 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67feb.aXhy7LTxZykCFMNi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8457 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881255 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fef.bDOzVs5KDFd77ioF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8462 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881261 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff1.pektwTPtRVEknIpF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8464 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881264 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff2.4SUHFty6sPWMoGzk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8465 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881265 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff4.lVhXyYHogfEX4dN7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8467 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881263 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff8.Qzc2AvhD3D+ln0l9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8474 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881271 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff4.xN/Lh8HeW6wG2fpO%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8468 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881262 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff7.MeJE3ipzNdzXcBiw%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8473 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881270 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff4.LE3YSR48VSh3Cel5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8466 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881266 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fee.b1MOE0SqRCAGnYJc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8461 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881260 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff5.9NGIt4wLMECOd3de%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8470 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881267 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff6.g2orCX0M17urfiJM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8472 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881269 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ffa.NFG6t8wp7gKwlQsm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8475 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881272 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff5.a7UIxG9+4VajcdVz%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8469 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881267 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff0.DDZvHQCRV7oeHSGK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8463 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881262 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ff6.taeIxgIZNqK73eBC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8471 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881268 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fff.r4u06PQuxZQ9+asU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8481 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881277 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ffb.MMksEA3xI9ZU7MP7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8476 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881273 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ffb.OKS6blLwroLTXnAy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8477 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881272 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68005.evUUrGACQkPgoWrX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8489 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881284 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68009.kHBi2iuYC0x62mrh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8493 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881284 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ffd.2wqvoqYybT658gWY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8479 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881276 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68000.pj0LHdD/5z5sduaN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8483 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881278 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68002.rIFFtmb0VXEMBrNU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8485 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881280 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ffc.VvTZowq7v4C4l2vI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8478 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881274 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fff.E+VgnSHkLVz564j/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8482 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881277 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68007.xry3TfA/ViUfhKH4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8490 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881285 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6800c.c2pGKyK7eTAJmxC8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8497 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881290 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67ffe.wHuSBlv5+KGa1fv0%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8480 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881275 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68002.213BJFBCjjSJHKjg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8486 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881281 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68008.PVY9cQb2C/jzRrhN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8492 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881286 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68009.BUEEqdUae8lQEKdi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8494 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881288 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6800a.wWQOQn3yEd1VAWeS%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8495 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881288 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6800e.GeeeGtgYHE1HKxtn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8499 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881293 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68010.OGfpcnifwtXA4Ohl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8501 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881295 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68005.WAlkfIZ97RqC0P1f%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8488 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881282 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68000.rqQb+8LUa+ISKZ/8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8484 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881279 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68003./Rshk2t+kJUXcatE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8487 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881282 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68007.sZvLVQWRe/Z+IR4q%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8491 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881286 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68013.TjkUOFcKumRxIFlb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8504 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881291 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68011.T7Ku6S9nVamYkLLY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8502 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881297 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68019.1+WT5gGhAyPH6cGy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8512 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881304 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6800d.X65Ta3a8dRSIVFEH%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8498 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881292 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68016.WSUDRExenLhV08Je%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8508 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881300 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6800f.F2YSZfa5j5LaEylD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8500 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881294 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6801d.tFoq8rxB7EjLVV//%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8515 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881308 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6801f.gIGThfBcPV+/dcHl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8516 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881310 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68020.LyCeVJiD1PQYXqdH%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8517 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881309 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68023.wdVuoZ3gujZFMbvP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8520 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881313 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68016.ffnNBY0EWKirz7wI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8509 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881302 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6801a.VUqwLCn2CcfrB3wx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8513 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881306 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6800b.ENPtpW6I4+J9m0Sn%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8496 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881289 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68014.bmzJd54xA58/ErcA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8506 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881299 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68019./GhA1llgSqPu/h2H%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8511 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881303 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68021.E2f+PLpiNY4nkrjT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8518 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881311 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68024.0Xb0JzAdWsgviHkh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8522 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881316 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68012.fK4jI1n1ebtBT2C2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8503 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881298 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68015./Ehnt0AlSlXElR2I%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8507 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881300 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68018.+XjGjVui+ywpdruD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8510 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881302 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68022./xsILqO8+VP7caMI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8519 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881313 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68023.QMGHQXov/mm5eyxN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8521 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881314 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6801b.NC9Pjk04uHlBwhs6%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8514 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881307 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68014.tKXV5ZEYDmvXuxDA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8505 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881298 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68028.HuA8mGvdy31gQGSY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8525 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881319 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6802d.Z12tx/FcKJjEcG/n%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8530 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881323 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68026.pqvSuJE/clKLfzr3%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8523 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881317 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68027.831ff014nwJvz89n%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8524 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881318 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68029.PRf3zWhHolgO1CeP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8526 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881321 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68030.GZT+jroigtqXQ21q%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8533 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881327 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68033.Ed76Q2TfBm+ZuwIx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8535 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881330 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68031.D9Bx+xpci0N6UxGR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8534 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881328 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68036.TL/ERf9v8380CP6V%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8537 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881333 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68038.V9pAg3M1omgWBWvL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8539 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881334 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6802a.JL2bG+EYrL+4p4An%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8527 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881320 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:43 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:43 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6802b.N4/2KA0w1ldZ7MsS%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8528 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881322 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:47 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6802f.iXNGgyVjAL5uAPFv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8532 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881325 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68036.dSNSQ5TOLYigd/fN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8538 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881334 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6802e.QwnntrZzU1A/tzA6%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8531 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881326 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68035.YZ+3yCRu+SEg92Ce%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8536 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881331 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68039.Hccf+7pnjuonqH8t%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8540 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881335 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:08:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:08:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6802d.+JsRUK4bVKLwBen7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8529 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881324 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6816b.QFMbXvG+jhB5d1wy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8851 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881642 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6816d.6s3HM5IIO8JLYIkB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8852 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881643 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6816f.9qAxTDupd4piXYIl%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8854 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881646 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6816d.x7S+6szAFSPdDE6Q%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8853 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881640 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68170.8BaRtltdewMgZV3L%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8855 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881645 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68171.LH2ZIznALI2Gz2P+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8858 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881648 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68176.w6ogP5/Yz8uK7IAU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8862 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881652 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68177.1U6/E3OayYZmQEQd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8864 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881654 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68170.E39sZNrzrnxArZ7L%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8856 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881647 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68171.rbgll+reA72nOHVe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8857 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881647 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6817d.gnZoAx7cGifWkKso%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8871 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881660 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68179.nBLbp5OOxfzQtXfs%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8866 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881656 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68178.E7HxVk5RSfuwhWSY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8865 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881655 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6817b.1qAZvGKEoVN0qtEP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8868 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881657 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6817c.6kYU7CYgqcd1z9cq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8869 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881659 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6817c.FOoD7ty3gGd/cWxL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8870 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881659 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68175.l4A6gDDjI58Uuk0p%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8861 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881652 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68172.CoydzJFXcM9niuJt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8859 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881649 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68174.fNTl/M1gDuiWMqpd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8860 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881651 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68177.pX6TjaTb8hDsUTEX%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8863 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881653 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68180.pcDap8zBH+hrNhMf%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8874 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881662 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68187.xk3QT/0fVHeCYZSi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8880 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881668 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68183.EKqClXlya09M8+Jt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8877 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881667 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6817f.xmxE5fKV2N1RAzmg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8873 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881663 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6817a.jtCwUuKdsxejOJ1y%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8867 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881656 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6817e.d49n9htzMczWehOZ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8872 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881661 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68180.OEM4V6abiDcfjtDj%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8875 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881663 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68189.IuoFFfR11Q3wwSDg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8882 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881670 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6818c.TPfeBwXorqn6JGXI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8886 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881674 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:49:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:49:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f7b.iJBaXiz9ncRcW9Mh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 283 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872954 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6818a.FgK3baihmtxY3GkE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8884 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881672 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6818b.24X+FD3DzzHRGhh+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8885 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881673 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:57:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:57:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66f92.lYR0xLI26oqWCAxe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4308 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877071 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:47:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:47:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f12.5yMiPT4DsjJ3Qgr1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 183 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872849 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68186.tpVYhsAQGzfHQ9Qm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8879 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881669 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:48:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:48:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f39.4RaCAnLvNLQGklEr%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 219 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872888 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:55:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:55:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66f14.U0XCm4igo58TDVvM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4177 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876947 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:56:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:56:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66f30.Txg84w2bPH3zeGH2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4207 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876975 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:05:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:05:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67f6f.PNkpfJm6znlInR+e%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8331 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881134 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68184.KhFmPhGzXmH1QMh+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8878 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881665 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:57:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:57:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d70.blzhoaWFUsi+rumr%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7838 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880623 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68188.9ZUvqNM1H66R7pqK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8881 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881670 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6818a.o/YQVvowSnzzr5Q1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8883 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881673 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:46:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:46:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ed6.GCJc5aBdqcc3jdMp%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 129 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872789 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:48:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:48:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f53.+Uf2rXY1MXRcmk98%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 246 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872914 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:48:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:48:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f60.bLBxVqCAsUF1qzG0%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 258 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872927 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:58:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:58:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66f9c.0/M+GL9Y2Tw1YtLK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4318 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877083 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:14:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:14:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68182.Skxy1KQDzDdJoZTs%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8876 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881664 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:05:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:05:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67f53.+NCpHLnIaUgWUvJ9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8304 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881105 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:35:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:35:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6784d.0RZrRxNxKwq6si4r%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6524 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879302 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:58:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:58:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66fc0.dHjmjm2+2fzbYOzN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4355 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877119 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:55:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:55:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66f0d.9NafWtCokc1CQZsK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4170 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876940 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:55 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:55 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d2b.is9uHY5rT3Q9AohL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7769 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880553 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:57:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:57:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d77.xs7eL0x44AK7iks1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7844 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880630 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:48:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:48:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f30.A0HjI/Sfzl+AB87f%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 211 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872880 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:49:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:49:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f8c.h8oVX+tMHLkQZnsw%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 299 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872971 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:35:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:35:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6787c.0OzCJQBshZNRcJc0%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6570 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879355 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:47:10 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:47:10 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65efe.49BbL4d8mcFLJ1ZA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 165 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872828 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:57:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:57:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d7d.+Y6K21Z/PfV4APMy%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7850 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880637 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:58:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:58:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67dcf.KDZRvx4mjUDcYA07%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7928 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880718 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:45:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:45:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ead.V05TjazyyhqQZs/O%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 90 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872749 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:05:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:05:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67f6c.cAzoXBSm639qOaJP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8327 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881131 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:38:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:38:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67928.48GDMe3vCqVjrzJa%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6740 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879525 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:56:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:56:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d34.HoHizH6vPzALRdwx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7779 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880562 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:57:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:57:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d7e.jwnpANR6so3HMsMc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7852 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880629 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:47:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:47:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f03.QwmoEhlNJzkR2TCp%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 169 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872833 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:09:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:09:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67252.faGPBwMrU/HM+zNY%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5013 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877777 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:47:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:47:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f1c.fsOtrrRpR86BSAVq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 190 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872859 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fd2.pTO8Zch/K14poaJg%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8430 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881227 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:37:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:37:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf678f5.Kt4xHgailR7EkveR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6686 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879475 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:35:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:35:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6785b.3l1cXN7UQSfk8Db9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6538 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879320 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:47:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:47:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f25./Eyk++oOK9qFvuSG%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 199 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872868 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:46:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:46:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ec0.fp9pCVUJGjrtY3Qq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 110 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872768 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:48:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:48:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f34.ckrYnrShbHr8zp3A%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 215 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872883 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:48:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:48:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f58.BDYtop+R3bQX1PnF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 249 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872917 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:50 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:50 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d26.NRauJljtsN3ZJUrN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7765 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880548 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:05:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:05:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67f5c./EVXOzm62p9IOSWa%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8312 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881115 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:57:56 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:57:56 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66f94.Y5QcMveyFwC7or7k%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4310 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877074 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:47:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:47:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65ef9.DYNXgrUUXcXrnB6A%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 160 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872823 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:59:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:59:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67e00.O1G6Z7Zs01unjfne%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7977 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880766 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:07:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:07:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fd7.AXpBEH8C4f/AN2bM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8437 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881238 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:47:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:47:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f04.dW3Cx18CgaR6mmS/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 171 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872835 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:58:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:58:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67dde.f0XWDSiZ/K+oNFgT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7942 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880732 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:36:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:36:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67884.WP6BCK086l0HMCQE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6578 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879363 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:19:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:19:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf674aa.KwtU+Kw6yV5N9TFc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 5613 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542878376 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:05:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:05:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67f51.RU7dXY2rBNVZqUlx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8302 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881102 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:55:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:55:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d12./p/0gLkR+J5px57h%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7746 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880528 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:48:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:48:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f38.tkz189aMz5BnS/yt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 218 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872887 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:47:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:47:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f08.MgqlJ8Z6/0pELoC7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 174 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872838 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:05:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:05:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67f4d.HPdFlGPIlkT+RrnM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8298 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881099 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:58:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:58:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66fa4.VRv0orknLL/tI9YD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4326 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877092 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:49:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:49:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf65f6c.Y3fprIew3Ez9JtU/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 271 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542872939 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:35:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:35:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6786d.8Rtw9TL33e2qHYFh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6554 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879339 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:56:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:56:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67d39.LpDy/z9ywT3LBEJb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 7783 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542880567 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:36:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:36:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf678ac.10H8/o+ajhQeT8du%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6618 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879403 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:06:47 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:06:47 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67fb7.jKqYICHbTILOVgkS%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8405 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881206 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:49:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:49:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689b8.mNMuALnhAtOFvjQe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10977 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883767 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:49:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:49:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689b9.aB0nhioSXmwb9+pa%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10978 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883766 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:49:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:49:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689bd.cZrik4GsJ4MC8Z0k%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10982 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883772 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:49:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:49:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689bc.2gqPOiipY8ybAya5%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10981 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883755 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:49:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:49:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689bf.SEegl84KJaJ8V/Co%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10984 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883774 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:49:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:49:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689ba.mj7VHTT9Cf91ktBu%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10979 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883768 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:49:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:49:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689c0.P38kLwjhAPXZ9/He%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10986 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883775 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:23:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:23:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66772.duqbWBrAlfpXDl/8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2258 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874993 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:49:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:49:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689bb.7JWYj7nbrqHJFARc%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10980 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883770 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:49:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:49:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689be.H3J4rNRkXVC3lesb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10983 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883771 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:21:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:21:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6671c.YKzh/alVJgBKr4mv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2173 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874906 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:49:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:49:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689bf.PBavT051tTOtLclJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10985 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883773 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:24:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:24:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf667a6.Z/JnQZCosLzDX7Uv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2306 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875045 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666d3.INuUUaBv2RL9rBvB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2097 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874834 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:13:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:13:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68140.PP4IIflDasWaGPyR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8807 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881598 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:33:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:33:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf677e2.6Ge1Zliqw5s7wl7L%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6417 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879199 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:30:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:30:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68558.0VB7JoPtqHCSjOYC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 9848 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882647 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666c2.HgsVenT8rtNPoFV4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2078 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874816 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:19:52 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:19:52 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666a8.+ZC7E3COKEmFvqfW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2054 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874790 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:22:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:22:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66755.iPhfPXQukXGQsqch%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2229 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874964 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:40:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:40:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf687ac.i2blcbubfclLEHWs%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10441 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883243 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:39:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:39:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68775.6BctpavU07a4rt4I%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10391 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883188 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666cb.mepTcRbWMU6+JyzD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2089 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874826 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:32:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:32:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf685bc.35WbQrsDBwn72JV9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 9944 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882746 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:31:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:31:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67784.62vJInHA+DOfEVEA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6327 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879105 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:40:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:40:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68785.M4abV05jmW1U/wF0%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10408 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883204 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:34:22 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:34:22 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6862e.CsWSzZrPC/nLx7IL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10060 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882861 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:30:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:30:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67723.x+RVGtUWk9HKZOJE%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6236 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879010 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:32:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:32:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf685ca.P2LWuANmYZe+O4J8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 9959 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882756 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:34:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:34:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68634.gl3q0I2b+85cEyu8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10066 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882866 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:39:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:39:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6877f.VzCHxacK+/tn53bW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10402 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883199 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:23:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:23:37 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66789.0PvzaHmj6ly81ixQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2281 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875017 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:34:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:34:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67821.ITHDz1szQVJCIUB8%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6481 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879264 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:34:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:34:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6863a.eHHEJtE5zTyziHBZ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10071 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882873 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:40:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:40:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf687a7.4OJgT3dFHJb6xYeC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10436 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883236 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:40:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:40:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6878b.wi2hHrOH29hmVy+n%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10410 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883205 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:12:31 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:12:31 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6810f.aAxcacVObKu0GN9H%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8759 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881550 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:41:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:41:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf687cb.XTKb0ifgnEN1isPv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10475 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883274 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:33:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:33:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf677de.HJg3HouUAp4U34Zs%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6413 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879196 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:34:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:34:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6780c.4NXdBs4vE7WBl4HF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6459 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879243 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:30:15 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:30:15 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67727.HYKdNE+yAAlMhOma%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6240 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879014 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:30:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:30:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68550.RoCYaJVKzBXxTrOD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 9841 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882639 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:32:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:32:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf685c3.r/48vyFdJTNNYpUq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 9952 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882752 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:11:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:11:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf680d8.KGCdEj8P/JRErzT+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8702 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881494 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:40:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:40:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68798.kTbOsYNk3JWnQXuh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10422 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883221 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:41:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:41:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf687c9.DvROe1JlHOcZKYo7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10472 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883271 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:30:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:30:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6855d.ZQ/r8NQxFx1uLrRf%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 9851 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882652 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:32:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:32:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf685bb.otGTpI6MryxECI46%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 9943 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882745 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666da.dlwVYewHTH/m04OV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2104 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874841 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:12:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:12:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf680f9.icnlpkiuGqkNIhiF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8734 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881528 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:09:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:09:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68072.wlMAY1LlyH3IVuRx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8598 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881393 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:42:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:42:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68833.pZvFodm/b4kYC9P/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10583 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883378 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:20:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:20:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf666cd.OQdQiwIHP0TKhY1f%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2091 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874828 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:33:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:33:38 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68602.hA3HCh8mVqDSMeaQ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10015 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882816 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:11:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:11:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf680d5.nJCSS0Vnu5yV0qVj%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8698 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881492 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:21:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:21:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66727.quoONy9GeIgWxfx/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2183 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542874918 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:13:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:13:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6812e.84wYVkmBUHglH2W0%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8791 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881577 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:33:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:33:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf677e1.H6HDRYF3QqH20YOF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 6416 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542879200 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:44:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:44:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6887d.3IdmY47UZTyEwtQx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10660 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883452 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:34:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:34:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6862c.65Fp3vJOsfxn13y2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10057 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882859 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:32:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:32:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf685a0.e/A56ptjgQHqa2p2%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 9920 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882718 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:39:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:39:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68774.gO4DPm+13ZmVS/nM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10390 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883188 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:42:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:42:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6880d.Ozrhg6c6rXyAyQ09%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 10544 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883340 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:29:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:29:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68519.NNZSiib0mDX4FWda%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 9788 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542882584 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:23:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:23:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6679d.aWxPpCtAQ5ilZngC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 2298 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542875036 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf692fc.gmwJHvu3kiI9gWJN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13288 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886139 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf692fe.T+w0jqFxh11Du7Ov%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13289 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886140 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:05 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:05 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69301.ileOTT0nlEFgTB98%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13291 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886144 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf692ff.B6Q73ePyih8MDW9f%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13290 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886141 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69302.BbcFRiRTPnARhHMS%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13292 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886144 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69305.hR1m4Aap5cjByh7l%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13295 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886148 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69303.8l6cJ4tzGSSXoFsM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13293 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886145 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69304.Hpf7LsTkx3dV/a4E%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13294 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886146 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6930a.e4Nd8qVsHApPgV10%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13300 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886153 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69308.AitebVdtdOVT8qOu%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13298 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886142 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:14 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:14 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6930a.QR8ZebAL1h43TqBV%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13301 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886151 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:17 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:17 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6930d.dQ287glvLUdeCBFt%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13303 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886155 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6930e.iCA1AcLPQrAGE2Pe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13304 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886157 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6930c.DxhVo9mq/NGPcVs3%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13302 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886154 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69309.M6sh6B858SDHMeOU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13299 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886151 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6930f.skrch5we41Z3lzpe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13305 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886158 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69307.pmV5dx9vcn5aDsVr%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13296 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886149 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69315.Wg2/bab4ibfjPlNf%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13311 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886163 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69318.WOIIERJffvGbCrZN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13314 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886167 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69311.cuLHVqK0ed2DeU2E%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13307 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886160 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69313.fysn8P4zNSflqzNS%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13309 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886154 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69308.1ouA6ptwXL78vFvC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13297 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886150 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69311.3boe1VUs/v74C7m4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13306 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886159 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:23 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:23 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69313.FV0bayoigTjaYeRi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13308 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886161 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:27 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:27 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69317.WncfwWlx/3Feg86V%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13313 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886166 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6931a.NN/kb485NDPbLZ56%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13316 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886169 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:29 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:29 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69319.8bVXvaE5d8lieG7P%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13315 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886168 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:33 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:33 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6931d.U6tUiTUrY2/rXpSN%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13319 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886172 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69320.dQ7EbKyJWB7dDJEb%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13322 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886174 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69314.o/fs+BoCafAXXQVM%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13310 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886133 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:34 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:34 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6931e.Yzsg5pUu3hCWUUk1%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13320 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886172 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6931f.5s7Bb7TmlutYm1ZU%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13321 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886173 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:12:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:12:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf680f7.tghA1qN1xNgbl3ol%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8731 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881525 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69315.kznfydNyYfSaLvhx%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13312 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886164 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:10:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:10:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68090.oXKaEDztvAn3QG17%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8629 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881423 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:20:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:20:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf690f0.k1t5c5qPweNYzWTq%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12784 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885597 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:13:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:13:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68148.fJoGeDNBZRpxde7t%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8814 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881600 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:12:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:12:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68f2a.m/veVV30GeIlRBWh%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12345 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885157 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:12:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:12:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf680f0.iYI9PCVqfUddS80i%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8724 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881519 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6931c.5mZROJjfH/DxgLd+%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13317 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886170 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:13:44 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:13:44 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68158.i7viFB6O1SDtfBQ4%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8833 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881622 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:50:28 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:50:28 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689f4.wwEGPCiOg4ji+Oh9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 11032 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883827 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:52:53 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:52:53 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68a85.yvwjMGC5Rrr7fTHP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 11170 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883971 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:02:26 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:02:26 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf670a2.eERy3/qAcMbOQRCe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4576 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877345 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:21:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:21:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69129.FFuQ3BXC/GmV2WPK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12837 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885670 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:04:18 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:04:18 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67112.0xrYSDBi/Os53j8G%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4690 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877457 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:20:39 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:20:39 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69107.Q7Bndipn92nYVZHK%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12804 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885636 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:00:03 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:00:03 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67013.+7/vHV1/S9Vgm2aB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4432 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877202 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:22:00 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:22:00 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69158.7j2tfpti3LqqnrGv%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12883 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885717 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:01:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:01:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67055.b8bHgJ1nRx1reEw9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4497 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877268 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:09:59 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:09:59 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68e87.qEX0bsQmvzOIQGg7%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12179 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542884998 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:10:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:10:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6808b.hgC7SBgBbKac1XWR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8623 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881418 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:12:21 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:12:21 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68105.pwNnTUHj2zFYam8u%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8747 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881540 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69320.xe3PtJKTfaKVcK8J%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13323 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886175 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:03:58 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:03:58 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf670fe.Lc/sUoJq9jEOx/IB%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4670 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877435 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:10:35 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:10:35 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6809b.uzJ4+D0ebSrng3Ow%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8640 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881434 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:11:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:11:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68ec8.JtoOuubk/8leWr5H%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12245 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885063 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:24:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:24:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf691d6.JnZUebK/mnU7KA4A%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13010 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885845 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:02:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:02:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67089.eyMF9+oMg3bpXxqd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4550 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877320 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:04:07 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:04:07 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67107.jE8OzRyH+G/D2ZrJ%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4679 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877437 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:50:02 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:50:02 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf689da.5WeTHndt46UT316Y%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 11011 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883801 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:11:01 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:11:01 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68ec5.KQ+7B05+ewDGp3Xd%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12241 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885058 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:02:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:02:45 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf670b5.2VwpJkr5kZUPtgAI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4595 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877364 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:59:57 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:59:57 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6700d.uetQn+/OjQ9VFRFL%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4425 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877192 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:02:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:02:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67094.NNz1FhY3ifG3uAx0%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4560 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877329 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 02:51:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 02:51:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6601a.7klRnZjuRJnYm8cR%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 433 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542873112 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:52:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:52:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68a59.RmRVyh3+j3k0JO0C%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 11127 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883929 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:20:40 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:20:40 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf69108.Ad0Jd7LlAWoB0EsP%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12806 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885638 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:10:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:10:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf680ae.n0F8zzk03qQrfmKo%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8661 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881452 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:54:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:54:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68ae6.mDEKV7RxOJuMlXZG%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 11264 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542884065 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:02:36 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:02:36 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf670ac.6VaLmeYqeaaULM6G%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4587 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877355 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:01:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:01:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67058.BAgCl1toqeBTCWQw%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4500 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877264 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:21:49 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:21:49 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6914d.wmpTFtjWgq0vJMsD%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12873 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885708 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:29:32 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:29:32 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6931c.aZy5g51BcArexhEm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 13318 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542886163 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:03:54 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:03:54 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf670fa.krw+a9py1dvwts43%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4665 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877432 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:13:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:13:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68132.biAdeEKX3F4RhN/3%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8794 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881582 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:01:25 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:01:25 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67065.F3cB9noKvCieFeEe%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4513 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877284 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:11:11 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:11:11 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf680bf.cISSF4CUTL/Rvdbi%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8681 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881470 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:13:13 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:13:13 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68f49.cQFSF68xX97x0khp%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12376 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885191 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:53:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:53:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68a9c.F0spnmJtXPs56MLC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 11189 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542883996 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:54:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:54:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68ace.YD8mvZo07syFIQgC%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 11241 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542884044 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:04:30 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:04:30 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6711e.TXZrM9ajJyTh2eVF%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4700 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877467 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:13:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:13:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68f4c.EnQjwwKSJPWMjuTI%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12378 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885193 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:12:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:12:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf680f8.JaCB00yMKXn/K/yW%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8732 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881526 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:01:06 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:01:06 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67052.oD/wi3Gm7gNXKCqk%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4495 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877266 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 07:51:12 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 07:51:12 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6a640.MRce+TBKnsiTQ1Jm%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 18200 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542891071 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 07:53:04 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 07:53:04 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6a6b0.ohIAqQERXfUlUYmj%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 18312 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542891183 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 07:53:09 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 07:53:09 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6a6b5.xZMxNrpHyWkiro86%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 18317 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542891188 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 07:51:19 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 07:51:19 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6a647.cpY4Rx628Ioz63US%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 18207 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542891078 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 03:55:20 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 03:55:20 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf66ef8.ATrA9h0blYIigUqO%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4147 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542876920 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:06:08 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:06:08 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf67f90.bK/pTzXa05yeGVAw%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8365 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881166 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:10:46 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 06:10:46 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf68eb6.hv8OJTfZ4JEcybr/%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 12227 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542885043 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 05:11:24 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 05:11:24 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf680cc.D12DSK7Zg/+kauch%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 8690 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542881481 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 04:02:48 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 04:02:48 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf670b8.qXs5ckJmYdL91kV9%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 4597 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542877367 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 09:29:51 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 09:29:51 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6bd5f.CaCH5hv7PQOsux1y%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 24057 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542896990 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 09:30:16 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 09:30:16 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6bd78.R84THpNJd2D6uufj%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 24078 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542897012 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 09:29:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 09:29:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6bd56.rG/pAO55iUK3PRGT%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 24052 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542896981 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 09:29:42 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 09:29:42 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6bd56.+QjWj4wgJgPFWquA%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 24053 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542896980 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 09:30:41 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 22 Nov 2018 09:30:41 -0500 Subject: [abrt] : watch killed by SIGBUS Message-ID: <5bf6bd91.PirZ3klG285dZ+4s%user@localhost> reason: watch killed by SIGBUS cmdline: watch -n 1 nvidia-smi executable: /usr/bin/watch pid: 2034 pwd: /zfsauton/home/bpatra/FAMLI (deleted) hostname: gpu2.int.autonlab.org count: 24105 abrt_version: 2.1.11 analyzer: CCpp architecture: x86_64 global_pid: 2034 kernel: 3.10.0-862.2.3.el7.x86_64 last_occurrence: 1542897039 os_release: Springdale Linux release 7.6 (Verona) time: Thu 22 Nov 2018 02:44:12 AM EST type: CCpp uid: 1143 core_backtrace: :{ "signal": 7 :, "executable": "/usr/bin/watch" :, "stacktrace": : [ { "crash_thread": true : , "frames": : [ { "address": 140498204459232 : , "build_id_offset": 12512 : } : , { "address": 140498230332265 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236393 : , "function_name": "__run_exit_handlers" : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 140498230332343 : , "build_id": "ec3913e9f2ece9d9dd940ca63a8cc4f9c3f3adc2" : , "build_id_offset": 236471 : , "file_name": "/usr/lib64/libc-2.17.so" : } : , { "address": 4204619 : , "build_id": "fbd4001e497435ad8371bf0f472fc3589a837d0c" : , "build_id_offset": 10315 : , "function_name": "main" : , "file_name": "/usr/bin/watch" : } ] : } ] :} cgroup: :11:devices:/ :10:freezer:/ :9:pids:/user.slice :8:perf_event:/ :7:cpuset:/ :6:cpuacct,cpu:/ :5:blkio:/ :4:memory:/ :3:net_prio,net_cls:/ :2:hugetlb:/ :1:name=systemd:/user.slice/user-1143.slice/session-3549.scope environ: :MPI_INCLUDE=/usr/include/mpich-x86_64 :MANPATH=/usr/share/man/mpich:/usr/share/man:/usr/local/share/man :XDG_SESSION_ID=3549 :VIRTUALENVWRAPPER_SCRIPT=/zfsauton/home/bpatra/.local/bin/virtualenvwrapper.sh :VIRTUALENVWRAPPER_PROJECT_FILENAME=.project :HOSTNAME=gpu2.int.autonlab.org :SELINUX_ROLE_REQUESTED= :TERM=screen :SHELL=/bin/bash :HISTSIZE=100000 :'SSH_CLIENT=192.168.6.12 47567 22' :SELINUX_USE_CURRENT_RANGE= :CVSROOT=svnhub.int.autonlab.org:/var/CVS :QTDIR=/usr/lib64/qt-3.3 :QTINC=/usr/lib64/qt-3.3/include :SSH_TTY=/dev/pts/11 :MOSES_PATH=/zfsauton/home/bpatra/UnsupervisedMT/mosesdecoder/ :LC_ALL=en_US.UTF-8 :QT_GRAPHICSSYSTEM_CHECKED=1 :USER=bpatra :MPI_PYTHON_SITEARCH=/usr/lib64/python2.7/site-packages/mpich :LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36: :LD_LIBRARY_PATH=/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/zfsauton/home/bpatra/local/lib64:/zfsauton/home/bpatra/local/lib:/usr/local/cuda-8.0/lib64:/usr/local/cuda/lib64:/usr/lib64/mpich/lib :MPI_LIB=/usr/lib64/mpich/lib :WORKON_HOME=/home/scratch/bpatra/envs :TMUX=/tmp/tmux-1143/default,844,5 :PATH=/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/zfsauton/home/bpatra/local/bin:/zfsauton/home/bpatra/.local/bin:/usr/local/cuda/bin:/usr/lib64/qt-3.3/bin:/usr/lib64/mpich/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin :MAIL=/var/spool/mail/bpatra :MPI_BIN=/usr/lib64/mpich/bin :VIRTUALENVWRAPPER_HOOK_DIR=/home/scratch/bpatra/envs :MPI_COMPILER=mpich-x86_64 :PWD=/zfsauton/home/bpatra/FAMLI :_LMFILES_=/etc/modulefiles/mpi/mpich-x86_64 :SVN=svn://svnhub.int.autonlab.org/psycho/auton :LANG=en_US.UTF-8 :MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/usr/local/share/Modules/modulefiles:/opt/share/Modules/modulefiles :LOADEDMODULES=mpi/mpich-x86_64 :jrmoniz=jrmoniz at bash.autonlab.org :TMUX_PANE=%25 :SELINUX_LEVEL_REQUESTED= :'PS1=\\[\\033[0;35m\\]\\u:\\[\\033[0;31m\\]\\W>\\[\x1b[m\x0f\\]' :HISTCONTROL=ignoredups :MPI_SYSCONFIG=/etc/mpich-x86_64 :local_py=/home/scratch/bpatra/envs/env1/bin/activate :SHLVL=2 :HOME=/zfsauton/home/bpatra :MPI_SUFFIX=_mpich :LS_OPTIONS=--color=auto :MPI_MAN=/usr/share/man/mpich :PYTHONPATH=/usr/lib64/python2.7/site-packages/mpich :LOGNAME=bpatra :QTLIB=/usr/lib64/qt-3.3/lib :CVS_RSH=ssh :XDG_DATA_DIRS=/zfsauton/home/bpatra/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share :'SSH_CONNECTION=192.168.6.12 38640 192.168.6.101 22' :LC_CTYPE=UTF-8 :MPI_HOME=/usr/lib64/mpich :MPI_FORTRAN_MOD_DIR=/usr/lib64/gfortran/modules/mpich-x86_64 :MODULESb/pkgconfig :OMP_NUM_THREADS=1 :'LESSOPEN=||/usr/bin/lesspipe.sh %s' :VIRTUALENVWRAPPER_WORKON_CD=1 :MUSE_DATA=/zfsauton/home/bpatra/MUSE/data :XDG_RUNTIME_DIR=/run/user/1143 :CUDA_CACHE_PATH=/home/scratch/bpatra/computecache :THEANORC=/zfsauton/home/bpatra/.theano/.theanorc :SCRATCH=/home/scratch/bpatra :aws=ubuntu at ec2-18-218-5-121.us-east-2.compute.amazonaws.com :'BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`\n}' :_=/usr/bin/watch limits: :Limit Soft Limit Hard Limit Units :Max cpu time unlimited unlimited seconds :Max file size unlimited unlimited bytes :Max data size unlimited unlimited bytes :Max stack size 8388608 unlimited bytes :Max core file size 0 unlimited bytes :Max resident set unlimited unlimited bytes :Max processes 4096 1030574 processes :Max open files 1024 4096 files :Max locked memory 65536 65536 bytes :Max address space unlimited unlimited bytes :Max file locks unlimited unlimited locks :Max pending signals 1030574 1030574 signals :Max msgqueue size 819200 819200 bytes :Max nice priority 0 0 :Max realtime priority 0 0 :Max realtime timeout unlimited unlimited us maps: :00400000-00405000 r-xp 00000000 fd:00 202109838 /usr/bin/watch :00604000-00605000 r--p 00004000 fd:00 202109838 /usr/bin/watch :00605000-00606000 rw-p 00005000 fd:00 202109838 /usr/bin/watch :02271000-02338000 rw-p 00000000 00:00 0 [heap] :7fc84272a000-7fc848c54000 r--p 00000000 fd:00 201378657 /usr/lib/locale/locale-archive :7fc848c54000-7fc848c63000 r-xp 00000000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848c63000-7fc848e62000 ---p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e62000-7fc848e63000 r--p 0000e000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e63000-7fc848e64000 rw-p 0000f000 fd:00 1458146 /usr/lib64/libbz2.so.1.0.6 :7fc848e64000-7fc848e79000 r-xp 00000000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc848e79000-7fc849078000 ---p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849078000-7fc849079000 r--p 00014000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc849079000-7fc84907a000 rw-p 00015000 fd:00 125252 /usr/lib64/libz.so.1.2.7 :7fc84907a000-7fc849091000 r-xp 00000000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849091000-7fc849290000 ---p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849290000-7fc849291000 r--p 00016000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849291000-7fc849292000 rw-p 00017000 fd:00 6379411 /usr/lib64/libelf-0.172.so :7fc849292000-7fc8492f2000 r-xp 00000000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8492f2000-7fc8494f2000 ---p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f2000-7fc8494f3000 r--p 00060000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f3000-7fc8494f4000 rw-p 00061000 fd:00 57212 /usr/lib64/libpcre.so.1.2.0 :7fc8494f4000-7fc8494f8000 r-xp 00000000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8494f8000-7fc8496f7000 ---p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f7000-7fc8496f8000 r--p 00003000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f8000-7fc8496f9000 rw-p 00004000 fd:00 125305 /usr/lib64/libattr.so.1.1.0 :7fc8496f9000-7fc849710000 r-xp 00000000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849710000-7fc84990f000 ---p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc84990f000-7fc849910000 r--p 00016000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849910000-7fc849911000 rw-p 00017000 fd:00 57188 /usr/lib64/libpthread-2.17.so :7fc849911000-7fc849915000 rw-p 00000000 00:00 0 :7fc849915000-7fc84992b000 r-xp 00000000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc84992b000-7fc849b2a000 ---p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2a000-7fc849b2b000 r--p 00015000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2b000-7fc849b2c000 rw-p 00016000 00:2b 109577843 /zfsauton/home/bpatra/local/lib64/libgcc_s.so.1 :7fc849b2c000-7fc849b78000 r-xp 00000000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849b78000-7fc849d78000 ---p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d78000-7fc849d7a000 r--p 0004c000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7a000-7fc849d7b000 rw-p 0004e000 fd:00 1356153 /usr/lib64/libdw-0.172.so :7fc849d7b000-7fc849d91000 r-xp 00000000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849d91000-7fc849f90000 ---p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f90000-7fc849f91000 r--p 00015000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f91000-7fc849f92000 rw-p 00016000 fd:00 1372556 /usr/lib64/libresolv-2.17.so :7fc849f92000-7fc849f94000 rw-p 00000000 00:00 0 :7fc849f94000-7fc849f98000 r-xp 00000000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc849f98000-7fc84a197000 ---p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a197000-7fc84a198000 r--p 00003000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a198000-7fc84a199000 rw-p 00004000 fd:00 134625 /usr/lib64/libgpg-error.so.0.10.0 :7fc84a199000-7fc84a216000 r-xp 00000000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a216000-7fc84a415000 ---p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a415000-7fc84a416000 r--p 0007c000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a416000-7fc84a419000 rw-p 0007d000 fd:00 134629 /usr/lib64/libgcrypt.so.11.8.2 :7fc84a419000-7fc84a41a000 rw-p 00000000 00:00 0 :7fc84a41a000-7fc84a42e000 r-xp 00000000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a42e000-7fc84a62d000 ---p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62d000-7fc84a62e000 r--p 00013000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62e000-7fc84a62f000 rw-p 00014000 fd:00 594934 /usr/lib64/liblz4.so.1.7.5 :7fc84a62f000-7fc84a654000 r-xp 00000000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a654000-7fc84a853000 ---p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a853000-7fc84a854000 r--p 00024000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a854000-7fc84a855000 rw-p 00025000 fd:00 125260 /usr/lib64/liblzma.so.5.2.2 :7fc84a855000-7fc84a879000 r-xp 00000000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84a879000-7fc84aa78000 ---p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa78000-7fc84aa79000 r--p 00023000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa79000-7fc84aa7a000 rw-p 00024000 fd:00 57205 /usr/lib64/libselinux.so.1 :7fc84aa7a000-7fc84aa7c000 rw-p 00000000 00:00 0 :7fc84aa7c000-7fc84aa83000 r-xp 00000000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84aa83000-7fc84ac82000 ---p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac82000-7fc84ac83000 r--p 00006000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac83000-7fc84ac84000 rw-p 00007000 fd:00 1372568 /usr/lib64/librt-2.17.so :7fc84ac84000-7fc84ad85000 r-xp 00000000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84ad85000-7fc84af84000 ---p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af84000-7fc84af85000 r--p 00100000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af85000-7fc84af86000 rw-p 00101000 fd:00 1350234 /usr/lib64/libm-2.17.so :7fc84af86000-7fc84af8a000 r-xp 00000000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84af8a000-7fc84b189000 ---p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b189000-7fc84b18a000 r--p 00003000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18a000-7fc84b18b000 rw-p 00004000 fd:00 125309 /usr/lib64/libcap.so.2.22 :7fc84b18b000-7fc84b34d000 r-xp 00000000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b34d000-7fc84b54d000 ---p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b54d000-7fc84b551000 r--p 001c2000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b551000-7fc84b553000 rw-p 001c6000 fd:00 1350226 /usr/lib64/libc-2.17.so :7fc84b553000-7fc84b558000 rw-p 00000000 00:00 0 :7fc84b558000-7fc84b55a000 r-xp 00000000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b55a000-7fc84b75a000 ---p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75a000-7fc84b75b000 r--p 00002000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75b000-7fc84b75c000 rw-p 00003000 fd:00 1350232 /usr/lib64/libdl-2.17.so :7fc84b75c000-7fc84b781000 r-xp 00000000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b781000-7fc84b981000 ---p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b981000-7fc84b985000 r--p 00025000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b985000-7fc84b986000 rw-p 00029000 fd:00 60039 /usr/lib64/libtinfo.so.5.9 :7fc84b986000-7fc84b9bd000 r-xp 00000000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84b9bd000-7fc84bbbc000 ---p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbc000-7fc84bbbd000 r--p 00036000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbd000-7fc84bbbe000 rw-p 00037000 fd:00 59999 /usr/lib64/libncursesw.so.5.9 :7fc84bbbe000-7fc84bbed000 r-xp 00000000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bbed000-7fc84bded000 ---p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bded000-7fc84bdee000 r--p 0002f000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdee000-7fc84bdef000 rw-p 00030000 fd:00 180074 /usr/lib64/libsystemd.so.0.6.0 :7fc84bdef000-7fc84be00000 r-xp 00000000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84be00000-7fc84c000000 ---p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c000000-7fc84c001000 r--p 00011000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c001000-7fc84c002000 rw-p 00012000 fd:00 360148 /usr/lib64/libprocps.so.4.0.0 :7fc84c002000-7fc84c016000 rw-p 00000000 00:00 0 :7fc84c016000-7fc84c038000 r-xp 00000000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c203000-7fc84c210000 rw-p 00000000 00:00 0 :7fc84c22f000-7fc84c236000 r--s 00000000 fd:00 70345986 /usr/lib64/gconv/gconv-modules.cache :7fc84c236000-7fc84c237000 rw-p 00000000 00:00 0 :7fc84c237000-7fc84c238000 r--p 00021000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c238000-7fc84c239000 rw-p 00022000 fd:00 57155 /usr/lib64/ld-2.17.so :7fc84c239000-7fc84c23a000 rw-p 00000000 00:00 0 :7ffe25bb5000-7ffe25bd7000 rw-p 00000000 00:00 0 [stack] :7ffe25bde000-7ffe25be0000 r-xp 00000000 00:00 0 [vdso] :ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] open_fds: :0:/dev/pts/49 :pos: 0 :flags: 0100002 :mnt_id: 23 os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_pid_status: :Name: watch :Umask: 0002 :State: S (sleeping) :Tgid: 2034 :Ngid: 0 :Pid: 2034 :PPid: 16382 :TracerPid: 0 :Uid: 1143 1143 1143 1143 :Gid: 1143 1143 1143 1143 :FDSize: 64 :Groups: 1143 :VmPeak: 157792 kB :VmSize: 157792 kB :VmLck: 0 kB :VmPin: 0 kB :VmHWM: 1420 kB :VmRSS: 1420 kB :RssAnon: 1052 kB :RssFile: 368 kB :RssShmem: 0 kB :VmData: 1000 kB :VmStk: 136 kB :VmExe: 20 kB :VmLib: 5736 kB :VmPTE: 124 kB :VmSwap: 0 kB :Threads: 1 :SigQ: 0/1030574 :SigPnd: 0000000000000000 :ShdPnd: 0000000000000000 :SigBlk: 0000000000000000 :SigIgn: 0000000000000000 :SigCgt: 0000000188084003 :CapInh: 0000000000000000 :CapPrm: 0000000000000000 :CapEff: 0000000000000000 :CapBnd: 0000001fffffffff :CapAmb: 0000000000000000 :Seccomp: 0 :Cpus_allowed: ffffff :Cpus_allowed_list: 0-23 :Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000003 :Mems_allowed_list: 0-1 :voluntary_ctxt_switches: 4 :nonvoluntary_ctxt_switches: 0 From auton.sysnotify at gmail.com Thu Nov 22 06:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Thu, 22 Nov 2018 06:00:00 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bf68c30.457c.250ba920@bhyve.int.autonlab.org> ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Fri Nov 23 05:03:29 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Fri, 23 Nov 2018 05:03:29 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181123100329.1E00C101B2E@gpu2.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Fri Nov 23 06:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Fri, 23 Nov 2018 06:00:00 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bf7ddb0.461a.1828e057@bhyve.int.autonlab.org> ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Fri Nov 23 10:07:25 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Fri, 23 Nov 2018 10:07:25 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181123150725.C402C11E972@ari.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Fri Nov 23 15:02:45 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Fri, 23 Nov 2018 15:02:45 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181123200245.923D2D0DB@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Sat Nov 24 05:11:19 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 24 Nov 2018 05:11:19 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181124101119.CE6C1101B44@gpu2.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Sat Nov 24 06:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 24 Nov 2018 06:00:00 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bf92f30.46bf.16ccf9a@bhyve.int.autonlab.org> ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Sat Nov 24 07:03:18 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 24 Nov 2018 07:03:18 -0500 Subject: Cron /usr/local/sbin/iocell send -r -u root -h uranus zroot Message-ID: <5bf93e06.46c8.164d0b3c@bhyve.int.autonlab.org> INFO: sending zroot/iocell to zroot/iocell: /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\ |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/- \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/ -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\| /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\done! From auton.sysnotify at gmail.com Sat Nov 24 10:09:49 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 24 Nov 2018 10:09:49 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181124150949.6C79A1606EE@ari.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Sat Nov 24 15:15:20 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 24 Nov 2018 15:15:20 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181124201520.57533D0E8@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Sat Nov 24 23:12:48 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sat, 24 Nov 2018 23:12:48 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181125041249.0E9B671FA4@ari.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not get metalink https://mirrors.fedoraproject.org/metalink?repo=epel-7&arch=x86_64 error was 14: HTTPS Error 503 - Service Unavailable From auton.sysnotify at gmail.com Sun Nov 25 05:13:30 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 25 Nov 2018 05:13:30 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181125101330.44286101B46@gpu2.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Sun Nov 25 06:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 25 Nov 2018 06:00:00 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bfa80b0.4760.55870771@bhyve.int.autonlab.org> ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Sun Nov 25 10:14:03 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 25 Nov 2018 10:14:03 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181125151403.2ADC11606E1@ari.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Sun Nov 25 16:13:30 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 25 Nov 2018 16:13:30 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181125211330.47FD5D100@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Sun Nov 25 22:00:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 25 Nov 2018 22:00:01 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 30 zroot/iocell/jails/600ec005-17f4-11e8-a693-0cc47aab0128/root Message-ID: <5bfb61b1.31c1.4aa7779b@warden.int.autonlab.org> There are 35 snapshots on 'zroot/iocell/jails/600ec005-17f4-11e8-a693-0cc47aab0128/root'. Can destroy 5 and keep 30. From auton.sysnotify at gmail.com Sun Nov 25 22:00:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 25 Nov 2018 22:00:01 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 50 zroot/iocell/jails/076a7105-0f6a-11e8-92df-0cc47a6baaa0/root Message-ID: <5bfb61b1.47c6.75cbdb12@bhyve.int.autonlab.org> There are 58 snapshots on 'zroot/iocell/jails/076a7105-0f6a-11e8-92df-0cc47a6baaa0/root'. Can destroy 8 and keep 50. From auton.sysnotify at gmail.com Sun Nov 25 22:30:02 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 25 Nov 2018 22:30:02 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 30 zroot/iocell/jails/b4b45c86-c4ed-11e7-b09c-0cc47a6baaa0/root Message-ID: <5bfb68ba.3223.9241b73@warden.int.autonlab.org> There are 35 snapshots on 'zroot/iocell/jails/b4b45c86-c4ed-11e7-b09c-0cc47a6baaa0/root'. Can destroy 5 and keep 30. From auton.sysnotify at gmail.com Sun Nov 25 22:30:01 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Sun, 25 Nov 2018 22:30:01 -0500 Subject: Cron /usr/local/sbin/zsd --destroy 10 --keep 50 zroot/iocell/jails/aea663c8-bf63-11e7-b4b4-0025900c0884/root Message-ID: <5bfb68b9.4a92.206b846e@bhyve.int.autonlab.org> There are 58 snapshots on 'zroot/iocell/jails/aea663c8-bf63-11e7-b4b4-0025900c0884/root'. Can destroy 8 and keep 50. From auton.sysnotify at gmail.com Mon Nov 26 06:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Mon, 26 Nov 2018 06:00:00 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bfbd230.4ac8.7106ef19@bhyve.int.autonlab.org> ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Mon Nov 26 06:08:40 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Mon, 26 Nov 2018 06:08:40 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181126110840.22B99101B3E@gpu2.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Mon Nov 26 11:13:13 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Mon, 26 Nov 2018 11:13:13 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181126161313.4BE134DE44@ari.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Mon Nov 26 15:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 26 Nov 2018 15:30:06 -0500 Subject: A desktop is down Message-ID: <20181126203006.7C63A412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 26 16:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 26 Nov 2018 16:00:06 -0500 Subject: A desktop is down Message-ID: <20181126210006.A5DA0412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Mon Nov 26 16:13:58 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Mon, 26 Nov 2018 16:13:58 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181126211358.99545D100@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Mon Nov 26 16:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Mon, 26 Nov 2018 16:30:06 -0500 Subject: A desktop is down Message-ID: <20181126213006.8680A412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 06:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Tue, 27 Nov 2018 06:00:00 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bfd23b0.4b66.5138822a@bhyve.int.autonlab.org> ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Tue Nov 27 07:02:57 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Tue, 27 Nov 2018 07:02:57 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181127120258.24E9C101B48@gpu2.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Tue Nov 27 10:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 10:00:05 -0500 Subject: A desktop is down Message-ID: <20181127150006.26443412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 11:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 11:00:06 -0500 Subject: A desktop is down Message-ID: <20181127160006.70E84412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 11:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 11:30:06 -0500 Subject: A desktop is down Message-ID: <20181127163006.959FB412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 12:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 12:00:06 -0500 Subject: A desktop is down Message-ID: <20181127170006.5EDEE412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 12:05:16 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Tue, 27 Nov 2018 12:05:16 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181127170516.D4D78763@ari.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Tue Nov 27 12:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 12:30:06 -0500 Subject: A desktop is down Message-ID: <20181127173006.ADA16412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 13:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 13:00:06 -0500 Subject: A desktop is down Message-ID: <20181127180006.4CD90412B48F@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 13:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 13:30:06 -0500 Subject: A desktop is down Message-ID: <20181127183007.56AFF412B48F@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 14:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 14:00:05 -0500 Subject: A desktop is down Message-ID: <20181127190005.D0844412B48F@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 14:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 14:30:06 -0500 Subject: A desktop is down Message-ID: <20181127193006.E6E27412B48F@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 15:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 15:00:06 -0500 Subject: A desktop is down Message-ID: <20181127200006.2A973412B48F@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 15:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 15:30:05 -0500 Subject: A desktop is down Message-ID: <20181127203006.2D388412B48F@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 16:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 16:00:06 -0500 Subject: A desktop is down Message-ID: <20181127210007.0CC93412B48F@lake.auton.cs.cmu.edu> line.auton.cs.cmu.edu loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 16:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 16:30:05 -0500 Subject: A desktop is down Message-ID: <20181127213006.30C75412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 17:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 17:00:06 -0500 Subject: A desktop is down Message-ID: <20181127220006.EFAB4412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 17:06:13 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Tue, 27 Nov 2018 17:06:13 -0500 (EST) Subject: Cron run-parts /etc/cron.hourly Message-ID: <20181127220613.4FD03171ED@gpu1.int.autonlab.org> /etc/cron.hourly/0yum-hourly.cron: Could not retrieve mirrorlist http://springdale.princeton.edu/data/springdale/SCL/7/x86_64/mirrorlist error was 14: HTTP Error 404 - Not Found From auton.sysnotify at gmail.com Tue Nov 27 17:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 17:30:06 -0500 Subject: A desktop is down Message-ID: <20181127223006.BCF35412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 18:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 18:00:06 -0500 Subject: A desktop is down Message-ID: <20181127230006.5AE88412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 18:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 18:30:06 -0500 Subject: A desktop is down Message-ID: <20181127233006.ED621412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 19:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 19:00:06 -0500 Subject: A desktop is down Message-ID: <20181128000006.E2E7A412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 19:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 19:30:06 -0500 Subject: A desktop is down Message-ID: <20181128003006.E7D7F412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 19:45:03 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Tue, 27 Nov 2018 19:45:03 -0500 Subject: Cron /usr/local/bin/rsnapshot -c /usr/local/etc/rsnapshot_desktops.conf daily Message-ID: <5bfde50f.1921.455b8e92@uranus.int.autonlab.org> ---------------------------------------------------------------------------- rsnapshot encountered an error! The program was invoked with these options: /usr/local/bin/rsnapshot -c /usr/local/etc/rsnapshot_desktops.conf daily ---------------------------------------------------------------------------- ERROR: /usr/bin/rsync returned 12 while processing backup at loom.auton.cs.cmu.edu:/home/awertz/ From auton.sysnotify at gmail.com Tue Nov 27 20:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 20:00:06 -0500 Subject: A desktop is down Message-ID: <20181128010006.EBA1B412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 20:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 20:30:06 -0500 Subject: A desktop is down Message-ID: <20181128013007.1DCA5412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 21:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 21:00:06 -0500 Subject: A desktop is down Message-ID: <20181128020006.B0E0A412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 21:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 21:30:06 -0500 Subject: A desktop is down Message-ID: <20181128023007.52265412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 22:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 22:00:05 -0500 Subject: A desktop is down Message-ID: <20181128030006.32CCA412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 22:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 22:30:06 -0500 Subject: A desktop is down Message-ID: <20181128033006.C83C6412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 23:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 23:00:06 -0500 Subject: A desktop is down Message-ID: <20181128040006.D1EF8412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Tue Nov 27 23:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Tue, 27 Nov 2018 23:30:06 -0500 Subject: A desktop is down Message-ID: <20181128043006.7C21E412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 00:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 00:00:06 -0500 Subject: A desktop is down Message-ID: <20181128050007.2CFE6412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 00:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 00:30:06 -0500 Subject: A desktop is down Message-ID: <20181128053006.6D448412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 01:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 01:00:06 -0500 Subject: A desktop is down Message-ID: <20181128060006.46D51412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 01:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 01:30:05 -0500 Subject: A desktop is down Message-ID: <20181128063006.3A425412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 02:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 02:00:06 -0500 Subject: A desktop is down Message-ID: <20181128070007.07D41412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 02:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 02:30:06 -0500 Subject: A desktop is down Message-ID: <20181128073006.6D6FC412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 03:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 03:00:05 -0500 Subject: A desktop is down Message-ID: <20181128080006.44130412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 03:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 03:30:05 -0500 Subject: A desktop is down Message-ID: <20181128083006.6B240412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 04:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 04:00:06 -0500 Subject: A desktop is down Message-ID: <20181128090006.C23E3412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 04:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 04:30:05 -0500 Subject: A desktop is down Message-ID: <20181128093006.0E503412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 05:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 05:00:06 -0500 Subject: A desktop is down Message-ID: <20181128100006.7F18E412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 05:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 05:30:06 -0500 Subject: A desktop is down Message-ID: <20181128103006.A16BF412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 06:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 06:00:06 -0500 Subject: A desktop is down Message-ID: <20181128110006.697DC412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 06:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Wed, 28 Nov 2018 06:00:00 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bfe7530.4c03.97f4adc@bhyve.int.autonlab.org> ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Wed Nov 28 06:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 06:30:06 -0500 Subject: A desktop is down Message-ID: <20181128113006.8C946412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 07:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 07:00:06 -0500 Subject: A desktop is down Message-ID: <20181128120006.4F850412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 07:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Wed, 28 Nov 2018 07:00:00 -0500 Subject: Cron /usr/local/sbin/iocell send -r -u root -h backup1 zroot Message-ID: <5bfe8340.4c0c.4454eb96@bhyve.int.autonlab.org> ERROR: please set up publickey authentication for root at backup1 From auton.sysnotify at gmail.com Wed Nov 28 07:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 07:30:05 -0500 Subject: A desktop is down Message-ID: <20181128123006.2CBC3412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 08:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 08:00:06 -0500 Subject: A desktop is down Message-ID: <20181128130007.0B53F412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 08:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 08:30:05 -0500 Subject: A desktop is down Message-ID: <20181128133005.DF418412B48F@lake.auton.cs.cmu.edu> loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 09:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 09:00:06 -0500 Subject: A desktop is down Message-ID: <20181128140006.7B938412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 09:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 09:30:06 -0500 Subject: A desktop is down Message-ID: <20181128143006.3E1E9412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 10:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 10:00:06 -0500 Subject: A desktop is down Message-ID: <20181128150006.C4D26412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 10:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 10:30:06 -0500 Subject: A desktop is down Message-ID: <20181128153007.35162412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 11:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 11:00:05 -0500 Subject: A desktop is down Message-ID: <20181128160005.DC633412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 11:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 11:30:05 -0500 Subject: A desktop is down Message-ID: <20181128163006.7024C412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 12:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 12:00:05 -0500 Subject: A desktop is down Message-ID: <20181128170006.298DF412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 12:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 12:30:05 -0500 Subject: A desktop is down Message-ID: <20181128173006.305A9412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 13:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 13:00:06 -0500 Subject: A desktop is down Message-ID: <20181128180007.34333412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 13:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 13:30:06 -0500 Subject: A desktop is down Message-ID: <20181128183006.A4963412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 14:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 14:00:06 -0500 Subject: A desktop is down Message-ID: <20181128190006.4D22D412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 14:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 14:30:06 -0500 Subject: A desktop is down Message-ID: <20181128193006.8CDC2412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 15:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 15:00:05 -0500 Subject: A desktop is down Message-ID: <20181128200006.35B7C412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 15:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 15:30:05 -0500 Subject: A desktop is down Message-ID: <20181128203006.0E49B412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 16:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 16:00:06 -0500 Subject: A desktop is down Message-ID: <20181128210007.5051C412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 16:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 16:30:06 -0500 Subject: A desktop is down Message-ID: <20181128213006.E3894412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 17:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 17:00:06 -0500 Subject: A desktop is down Message-ID: <20181128220006.57DF9412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 17:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 17:30:06 -0500 Subject: A desktop is down Message-ID: <20181128223006.4FCC3412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 18:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 18:00:06 -0500 Subject: A desktop is down Message-ID: <20181128230006.C27E1412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 18:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 18:30:06 -0500 Subject: A desktop is down Message-ID: <20181128233006.6960A412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 19:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 19:00:06 -0500 Subject: A desktop is down Message-ID: <20181129000006.E1357412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 19:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 19:30:06 -0500 Subject: A desktop is down Message-ID: <20181129003006.BD68D412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 20:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 20:00:06 -0500 Subject: A desktop is down Message-ID: <20181129010006.590B5412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 20:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 20:30:06 -0500 Subject: A desktop is down Message-ID: <20181129013006.95F4C412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 21:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 21:00:06 -0500 Subject: A desktop is down Message-ID: <20181129020006.A51B8412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 21:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 21:30:05 -0500 Subject: A desktop is down Message-ID: <20181129023006.08498412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 22:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 22:00:06 -0500 Subject: A desktop is down Message-ID: <20181129030006.937F2412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 22:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 22:30:06 -0500 Subject: A desktop is down Message-ID: <20181129033006.C78DC412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 23:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 23:00:06 -0500 Subject: A desktop is down Message-ID: <20181129040006.98962412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Wed Nov 28 23:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Wed, 28 Nov 2018 23:30:06 -0500 Subject: A desktop is down Message-ID: <20181129043006.75FFF412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 00:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 00:00:06 -0500 Subject: A desktop is down Message-ID: <20181129050006.89D89412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 00:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 00:30:06 -0500 Subject: A desktop is down Message-ID: <20181129053006.B4BE2412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 01:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 01:00:05 -0500 Subject: A desktop is down Message-ID: <20181129060006.77D20412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 01:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 01:30:05 -0500 Subject: A desktop is down Message-ID: <20181129063006.1158E412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 02:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 02:00:06 -0500 Subject: A desktop is down Message-ID: <20181129070006.EBB24412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 02:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 02:30:06 -0500 Subject: A desktop is down Message-ID: <20181129073006.D008E412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 03:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 03:00:06 -0500 Subject: A desktop is down Message-ID: <20181129080006.7458D412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 03:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 03:30:05 -0500 Subject: A desktop is down Message-ID: <20181129083006.6786E412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 04:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 04:00:05 -0500 Subject: A desktop is down Message-ID: <20181129090005.ED7EB412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 04:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 04:30:05 -0500 Subject: A desktop is down Message-ID: <20181129093006.1AC14412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 05:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 05:00:06 -0500 Subject: A desktop is down Message-ID: <20181129100006.A5E34412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 05:30:11 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 05:30:11 -0500 Subject: A desktop is down Message-ID: <20181129103011.99888412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 06:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Thu, 29 Nov 2018 06:00:00 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5bffc6b0.4ca6.a4fe76e@bhyve.int.autonlab.org> ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Thu Nov 29 06:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 06:00:06 -0500 Subject: A desktop is down Message-ID: <20181129110007.13CFE412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 06:19:38 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 29 Nov 2018 06:19:38 -0500 (EST) Subject: *** SECURITY information for gpu1.int.autonlab.org *** Message-ID: <20181129111938.2494E355E5@gpu1.int.autonlab.org> gpu1.int.autonlab.org : Nov 29 06:19:38 : jrmoniz : user NOT in sudoers ; TTY=pts/13 ; PWD=/zfsauton/home/jrmoniz/ASGARD/datasets/visualgenome ; USER=root ; COMMAND=/bin/ln -s /zfsauton/home/jrmoniz/WeakerGraphBaselines/data/visualgenome/resnet_features_data resnet_features From auton.sysnotify at gmail.com Thu Nov 29 06:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 06:30:06 -0500 Subject: A desktop is down Message-ID: <20181129113007.38AA5412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 07:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 07:00:06 -0500 Subject: A desktop is down Message-ID: <20181129120006.D9C2D412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 07:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 07:30:05 -0500 Subject: A desktop is down Message-ID: <20181129123005.D71AD412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 08:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 08:00:06 -0500 Subject: A desktop is down Message-ID: <20181129130006.BFFA8412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 08:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 08:30:06 -0500 Subject: A desktop is down Message-ID: <20181129133006.99242412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 09:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 09:00:05 -0500 Subject: A desktop is down Message-ID: <20181129140006.39872412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 12:30:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 29 Nov 2018 12:30:37 -0500 Subject: SMART error (OfflineUncorrectableSector) detected on host: uranus Message-ID: <5c00223d.50.27f96162@uranus.int.autonlab.org> This message was generated by the smartd daemon running on: host name: uranus DNS domain: int.autonlab.org The following warning/error was logged by the smartd daemon: Device: /dev/da12 [SAT], 8 Offline uncorrectable sectors Device info: ST4000NM0024-1HT178, S/N:Z4F03FQ7, WWN:5-000c50-07b0cff9c, FW:SN02, 4.00 TB For details see host's SYSLOG. You can also use the smartctl utility for further investigation. No additional messages about this problem will be sent. From auton.sysnotify at gmail.com Thu Nov 29 12:30:37 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 29 Nov 2018 12:30:37 -0500 Subject: SMART error (CurrentPendingSector) detected on host: uranus Message-ID: <5c00223d.4e.6ba1fe52@uranus.int.autonlab.org> This message was generated by the smartd daemon running on: host name: uranus DNS domain: int.autonlab.org The following warning/error was logged by the smartd daemon: Device: /dev/da12 [SAT], 8 Currently unreadable (pending) sectors Device info: ST4000NM0024-1HT178, S/N:Z4F03FQ7, WWN:5-000c50-07b0cff9c, FW:SN02, 4.00 TB For details see host's SYSLOG. You can also use the smartctl utility for further investigation. No additional messages about this problem will be sent. From auton.sysnotify at gmail.com Thu Nov 29 13:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 13:30:06 -0500 Subject: A desktop is down Message-ID: <20181129183006.5F7BF412B48F@lake.auton.cs.cmu.edu> lama.auton.cs.cmu.edu land.auton.cs.cmu.edu lean.auton.cs.cmu.edu lena.auton.cs.cmu.edu line.auton.cs.cmu.edu lisa.auton.cs.cmu.edu litl.auton.cs.cmu.edu loco.auton.cs.cmu.edu loom.auton.cs.cmu.edu lori.auton.cs.cmu.edu luna.auton.cs.cmu.edu lyra.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 21:30:07 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 21:30:07 -0500 Subject: A desktop is down Message-ID: <20181130023007.42F60412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 22:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 22:00:05 -0500 Subject: A desktop is down Message-ID: <20181130030006.0DC3B412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 22:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 22:30:05 -0500 Subject: A desktop is down Message-ID: <20181130033005.D6193412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 23:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 23:00:06 -0500 Subject: A desktop is down Message-ID: <20181130040006.B5A87412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Thu Nov 29 23:15:45 2018 From: auton.sysnotify at gmail.com (auton.sysnotify at gmail.com) Date: Thu, 29 Nov 2018 23:15:45 -0500 Subject: [abrt] kernel: mce: [Hardware Error]: Machine check events logged Message-ID: <5c00b971.0A85pVPP5NbN8XuP%user@localhost> reason: mce: [Hardware Error]: Machine check events logged cmdline: BOOT_IMAGE=/vmlinuz-3.10.0-862.11.6.el7.x86_64 root=UUID=f47329d6-3b9a-419b-b11a-723d45086e59 ro crashkernel=auto rd.md.uuid=405aeaf1:88568fd8:ec8a357f:35f0e42f rd.md.uuid=d587da04:7ad06ea3:cd3e3baa:c7655e89 rhgb quiet LANG=en_US.UTF-8 package: kernel component: kernel hostname: ari.int.autonlab.org count: 1 abrt_version: 2.1.11 analyzer: Kerneloops architecture: x86_64 duphash: f1bc1e4e09509aa2a88d5b929737d611b52d2453 event_log: kernel: 3.10.0-862.11.6.el7.x86_64 last_occurrence: 1543551343 os_release: Springdale Linux release 7.6 (Verona) pkg_arch: x86_64 pkg_epoch: 0 pkg_fingerprint: 16CF C333 41A4 0948 pkg_name: kernel pkg_release: 862.11.6.el7 pkg_vendor: (none) pkg_version: 3.10.0 runlevel: N 3 time: Thu 29 Nov 2018 11:15:43 PM EST type: Kerneloops uid: 0 username: root uuid: f1bc1e4e09509aa2a88d5b929737d611b52d2453 backtrace: :The kernel log indicates that hardware errors were detected. :System log may have more information. :The last 20 mcelog lines of system log are: :========================================== :Nov 29 23:15:41 ari mcelog: STATUS 8c00004e000800c2 MCGSTATUS 0 :Nov 29 23:15:41 ari mcelog: MCGCAP 1000c19 APICID 40 SOCKETID 2 :Nov 29 23:15:41 ari mcelog: CPUID Vendor Intel Family 6 Model 62 :Nov 29 23:15:41 ari mcelog: Hardware event. This is not a software error. :Nov 29 23:15:41 ari mcelog: MCE 1 :Nov 29 23:15:41 ari mcelog: CPU 16 BANK 11 :Nov 29 23:15:41 ari mcelog: MISC 1229404000401c8c ADDR 5d7be59000 :Nov 29 23:15:41 ari mcelog: TIME 1543551341 Thu Nov 29 23:15:41 2018 :Nov 29 23:15:41 ari mcelog: MCG status: :Nov 29 23:15:41 ari mcelog: MCi status: :Nov 29 23:15:41 ari mcelog: Corrected error :Nov 29 23:15:41 ari mcelog: MCi_MISC register valid :Nov 29 23:15:41 ari mcelog: MCi_ADDR register valid :Nov 29 23:15:41 ari mcelog: MCA: MEMORY CONTROLLER MS_CHANNEL2_ERR :Nov 29 23:15:41 ari mcelog: Transaction: Memory scrubbing error :Nov 29 23:15:41 ari mcelog: MemCtrl: Corrected patrol scrub error :Nov 29 23:15:41 ari mcelog: :Nov 29 23:15:41 ari mcelog: STATUS 8c00004e000800c2 MCGSTATUS 0 :Nov 29 23:15:41 ari mcelog: MCGCAP 1000c19 APICID 40 SOCKETID 2 :Nov 29 23:15:41 ari mcelog: CPUID Vendor Intel Family 6 Model 62 dmesg: :[ 0.000000] microcode: microcode updated early to revision 0x42d, date = 2018-04-25 :[ 0.000000] Initializing cgroup subsys cpuset :[ 0.000000] Initializing cgroup subsys cpu :[ 0.000000] Initializing cgroup subsys cpuacct :[ 0.000000] Linux version 3.10.0-862.11.6.el7.x86_64 (mockbuild at cutlass.springdale.princeton.edu) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) ) #1 SMP Tue Aug 14 15:29:59 EDT 2018 :[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.10.0-862.11.6.el7.x86_64 root=UUID=f47329d6-3b9a-419b-b11a-723d45086e59 ro crashkernel=auto rd.md.uuid=405aeaf1:88568fd8:ec8a357f:35f0e42f rd.md.uuid=d587da04:7ad06ea3:cd3e3baa:c7655e89 rhgb quiet LANG=en_US.UTF-8 :[ 0.000000] e820: BIOS-provided physical RAM map: :[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009abff] usable :[ 0.000000] BIOS-e820: [mem 0x000000000009ac00-0x000000000009ffff] reserved :[ 0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved :[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007df8bfff] usable :[ 0.000000] BIOS-e820: [mem 0x000000007df8c000-0x000000007e01afff] reserved :[ 0.000000] BIOS-e820: [mem 0x000000007e01b000-0x000000007e36dfff] ACPI NVS :[ 0.000000] BIOS-e820: [mem 0x000000007e36e000-0x000000007f35ffff] reserved :[ 0.000000] BIOS-e820: [mem 0x000000007f360000-0x000000007f7fffff] ACPI NVS :[ 0.000000] BIOS-e820: [mem 0x0000000080000000-0x000000008fffffff] reserved :[ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed3ffff] reserved :[ 0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved :[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000807fffffff] usable :[ 0.000000] NX (Execute Disable) protection: active :[ 0.000000] SMBIOS 2.7 present. :[ 0.000000] DMI: Silicon Mechanics Rackform iServ R420.v4/X9QR7-TF+/X9QRi-F+, BIOS 3.0b 04/28/2015 :[ 0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved :[ 0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable :[ 0.000000] e820: last_pfn = 0x8080000 max_arch_pfn = 0x400000000 :[ 0.000000] MTRR default type: uncachable :[ 0.000000] MTRR fixed ranges enabled: :[ 0.000000] 00000-9FFFF write-back :[ 0.000000] A0000-BFFFF uncachable :[ 0.000000] C0000-CFFFF write-protect :[ 0.000000] D0000-E7FFF uncachable :[ 0.000000] E8000-FFFFF write-protect :[ 0.000000] MTRR variable ranges enabled: :[ 0.000000] 0 base 000000000000 mask 3F8000000000 write-back :[ 0.000000] 1 base 008000000000 mask 3FFF80000000 write-back :[ 0.000000] 2 base 000080000000 mask 3FFF80000000 uncachable :[ 0.000000] 3 disabled :[ 0.000000] 4 disabled :[ 0.000000] 5 disabled :[ 0.000000] 6 disabled :[ 0.000000] 7 disabled :[ 0.000000] 8 disabled :[ 0.000000] 9 disabled :[ 0.000000] PAT configuration [0-7]: WB WC UC- UC WB WP UC- UC :[ 0.000000] original variable MTRRs :[ 0.000000] reg 0, base: 0GB, range: 512GB, type WB :[ 0.000000] reg 1, base: 512GB, range: 2GB, type WB :[ 0.000000] reg 2, base: 2GB, range: 2GB, type UC :[ 0.000000] total RAM covered: 524288M :[ 0.000000] Found optimal setting for mtrr clean up :[ 0.000000] gran_size: 64K chunk_size: 512G num_reg: 3 lose cover RAM: 0G :[ 0.000000] New variable MTRRs :[ 0.000000] reg 0, base: 0GB, range: 512GB, type WB :[ 0.000000] reg 1, base: 2GB, range: 2GB, type UC :[ 0.000000] reg 2, base: 512GB, range: 2GB, type WB :[ 0.000000] e820: update [mem 0x80000000-0xffffffff] usable ==> reserved :[ 0.000000] e820: last_pfn = 0x7df8c max_arch_pfn = 0x400000000 :[ 0.000000] found SMP MP-table at [mem 0x000fde70-0x000fde7f] mapped at [ffffffffff200e70] :[ 0.000000] Base memory trampoline at [ffff938900094000] 94000 size 24576 :[ 0.000000] Using GB pages for direct mapping :[ 0.000000] BRK [0x653c42f000, 0x653c42ffff] PGTABLE :[ 0.000000] BRK [0x653c430000, 0x653c430fff] PGTABLE :[ 0.000000] BRK [0x653c431000, 0x653c431fff] PGTABLE :[ 0.000000] BRK [0x653c432000, 0x653c432fff] PGTABLE :[ 0.000000] BRK [0x653c433000, 0x653c433fff] PGTABLE :[ 0.000000] BRK [0x653c434000, 0x653c434fff] PGTABLE :[ 0.000000] BRK [0x653c435000, 0x653c435fff] PGTABLE :[ 0.000000] RAMDISK: [mem 0x35a97000-0x36d43fff] :[ 0.000000] Early table checksum verification disabled :[ 0.000000] ACPI: RSDP 00000000000f04a0 00024 (v02 ALASKA) :[ 0.000000] ACPI: XSDT 000000007e274098 000AC (v01 ALASKA A M I 01072009 AMI 00010013) :[ 0.000000] ACPI: FACP 000000007e2815e0 000F4 (v04 ALASKA A M I 01072009 AMI 00010013) :[ 0.000000] ACPI: DSDT 000000007e2741d8 0D403 (v02 ALASKA A M I 00000001 INTL 20051117) :[ 0.000000] ACPI: FACS 000000007e365f80 00040 :[ 0.000000] ACPI: APIC 000000007e2816d8 0023C (v03 ALASKA A M I 01072009 AMI 00010013) :[ 0.000000] ACPI: FPDT 000000007e281918 00044 (v01 ALASKA A M I 01072009 AMI 00010013) :[ 0.000000] ACPI: TCPA 000000007e281960 00032 (v02 APTIO4 NAPAASF 00000001 MSFT 01000013) :[ 0.000000] ACPI: SRAT 000000007e281998 00730 (v01 A M I AMI SRAT 00000001 AMI. 00000000) :[ 0.000000] ACPI: SLIT 000000007e2820c8 0003C (v01 A M I AMI SLIT 00000000 AMI. 00000000) :[ 0.000000] ACPI: HPET 000000007e282108 00038 (v01 ALASKA A M I 01072009 AMI. 00000005) :[ 0.000000] ACPI: PRAD 000000007e282140 000BE (v02 PRADID PRADTID 00000001 MSFT 03000001) :[ 0.000000] ACPI: SPMI 000000007e282200 00040 (v05 A M I OEMSPMI 00000000 AMI. 00000000) :[ 0.000000] ACPI: SSDT 000000007e282240 D0CB0 (v02 INTEL CpuPm 00004000 INTL 20051117) :[ 0.000000] ACPI: SPCR 000000007e352ef0 00050 (v01 A M I APTIO4 01072009 AMI. 00000005) :[ 0.000000] ACPI: EINJ 000000007e352f40 00130 (v01 AMI AMI EINJ 00000000 00000000) :[ 0.000000] ACPI: ERST 000000007e353070 00230 (v01 AMIER AMI ERST 00000000 00000000) :[ 0.000000] ACPI: HEST 000000007e3532a0 000A8 (v01 AMI AMI HEST 00000000 00000000) :[ 0.000000] ACPI: BERT 000000007e353348 00030 (v01 AMI AMI BERT 00000000 00000000) :[ 0.000000] ACPI: DMAR 000000007e353378 00230 (v01 A M I OEMDMAR 00000001 INTL 00000001) :[ 0.000000] ACPI: MCFG 000000007e3535a8 0003C (v01 ALASKA A M I 01072009 MSFT 00000097) :[ 0.000000] ACPI: Local APIC address 0xfee00000 :[ 0.000000] SRAT: PXM 0 -> APIC 0x00 -> Node 0 :[ 0.000000] SRAT: PXM 0 -> APIC 0x02 -> Node 0 :[ 0.000000] SRAT: PXM 0 -> APIC 0x04 -> Node 0 :[ 0.000000] SRAT: PXM 0 -> APIC 0x06 -> Node 0 :[ 0.000000] SRAT: PXM 0 -> APIC 0x08 -> Node 0 :[ 0.000000] SRAT: PXM 0 -> APIC 0x0a -> Node 0 :[ 0.000000] SRAT: PXM 0 -> APIC 0x0c -> Node 0 :[ 0.000000] SRAT: PXM 0 -> APIC 0x0e -> Node 0 :[ 0.000000] SRAT: PXM 1 -> APIC 0x20 -> Node 1 :[ 0.000000] SRAT: PXM 1 -> APIC 0x22 -> Node 1 :[ 0.000000] SRAT: PXM 1 -> APIC 0x24 -> Node 1 :[ 0.000000] SRAT: PXM 1 -> APIC 0x26 -> Node 1 :[ 0.000000] SRAT: PXM 1 -> APIC 0x28 -> Node 1 :[ 0.000000] SRAT: PXM 1 -> APIC 0x2a -> Node 1 :[ 0.000000] SRAT: PXM 1 -> APIC 0x2c -> Node 1 :[ 0.000000] SRAT: PXM 1 -> APIC 0x2e -> Node 1 :[ 0.000000] SRAT: PXM 2 -> APIC 0x40 -> Node 2 :[ 0.000000] SRAT: PXM 2 -> APIC 0x42 -> Node 2 :[ 0.000000] SRAT: PXM 2 -> APIC 0x44 -> Node 2 :[ 0.000000] SRAT: PXM 2 -> APIC 0x46 -> Node 2 :[ 0.000000] SRAT: PXM 2 -> APIC 0x48 -> Node 2 :[ 0.000000] SRAT: PXM 2 -> APIC 0x4a -> Node 2 :[ 0.000000] SRAT: PXM 2 -> APIC 0x4c -> Node 2 :[ 0.000000] SRAT: PXM 2 -> APIC 0x4e -> Node 2 :[ 0.000000] SRAT: PXM 3 -> APIC 0x60 -> Node 3 :[ 0.000000] SRAT: PXM 3 -> APIC 0x62 -> Node 3 :[ 0.000000] SRAT: PXM 3 -> APIC 0x64 -> Node 3 :[ 0.000000] SRAT: PXM 3 -> APIC 0x66 -> Node 3 :[ 0.000000] SRAT: PXM 3 -> APIC 0x68 -> Node 3 :[ 0.000000] SRAT: PXM 3 -> APIC 0x6a -> Node 3 :[ 0.000000] SRAT: PXM 3 -> APIC 0x6c -> Node 3 :[ 0.000000] SRAT: PXM 3 -> APIC 0x6e -> Node 3 :[ 0.000000] SRAT: Node 0 PXM 0 [mem 0x00000000-0x7fffffff] :[ 0.000000] SRAT: Node 0 PXM 0 [mem 0x100000000-0x207fffffff] :[ 0.000000] SRAT: Node 1 PXM 1 [mem 0x2080000000-0x407fffffff] :[ 0.000000] SRAT: Node 2 PXM 2 [mem 0x4080000000-0x607fffffff] :[ 0.000000] SRAT: Node 3 PXM 3 [mem 0x6080000000-0x807fffffff] :[ 0.000000] NUMA: Initialized distance table, cnt=4 :[ 0.000000] NUMA: Node 0 [mem 0x00000000-0x7fffffff] + [mem 0x100000000-0x207fffffff] -> [mem 0x00000000-0x207fffffff] :[ 0.000000] NODE_DATA(0) allocated [mem 0x207ffd9000-0x207fffffff] :[ 0.000000] NODE_DATA(1) allocated [mem 0x407ffd9000-0x407fffffff] :[ 0.000000] NODE_DATA(2) allocated [mem 0x607ffd9000-0x607fffffff] :[ 0.000000] NODE_DATA(3) allocated [mem 0x807ffd8000-0x807fffefff] :[ 0.000000] Reserving 192MB of memory at 656MB for crashkernel (System RAM: 524255MB) :[ 0.000000] Zone ranges: :[ 0.000000] DMA [mem 0x00001000-0x00ffffff] :[ 0.000000] DMA32 [mem 0x01000000-0xffffffff] :[ 0.000000] Normal [mem 0x100000000-0x807fffffff] :[ 0.000000] Movable zone start for each node :[ 0.000000] Early memory node ranges :[ 0.000000] node 0: [mem 0x00001000-0x00099fff] :[ 0.000000] node 0: [mem 0x00100000-0x7df8bfff] :[ 0.000000] node 0: [mem 0x100000000-0x207fffffff] :[ 0.000000] node 1: [mem 0x2080000000-0x407fffffff] :[ 0.000000] node 2: [mem 0x4080000000-0x607fffffff] :[ 0.000000] node 3: [mem 0x6080000000-0x807fffffff] :[ 0.000000] Initmem setup node 0 [mem 0x00001000-0x207fffffff] :[ 0.000000] On node 0 totalpages: 33546021 :[ 0.000000] DMA zone: 64 pages used for memmap :[ 0.000000] DMA zone: 21 pages reserved :[ 0.000000] DMA zone: 3993 pages, LIFO batch:0 :[ 0.000000] DMA32 zone: 7999 pages used for memmap :[ 0.000000] DMA32 zone: 511884 pages, LIFO batch:31 :[ 0.000000] Normal zone: 516096 pages used for memmap :[ 0.000000] Normal zone: 33030144 pages, LIFO batch:31 :[ 0.000000] Initmem setup node 1 [mem 0x2080000000-0x407fffffff] :[ 0.000000] On node 1 totalpages: 33554432 :[ 0.000000] Normal zone: 524288 pages used for memmap :[ 0.000000] Normal zone: 33554432 pages, LIFO batch:31 :[ 0.000000] Initmem setup node 2 [mem 0x4080000000-0x607fffffff] :[ 0.000000] On node 2 totalpages: 33554432 :[ 0.000000] Normal zone: 524288 pages used for memmap :[ 0.000000] Normal zone: 33554432 pages, LIFO batch:31 :[ 0.000000] Initmem setup node 3 [mem 0x6080000000-0x807fffffff] :[ 0.000000] On node 3 totalpages: 33554432 :[ 0.000000] Normal zone: 524288 pages used for memmap :[ 0.000000] Normal zone: 33554432 pages, LIFO batch:31 :[ 0.000000] ACPI: PM-Timer IO Port: 0x408 :[ 0.000000] ACPI: Local APIC address 0xfee00000 :[ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x0a] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x0c] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x0e] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x20] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x22] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x24] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x26] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x28] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x2a] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x2c] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x2e] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0x40] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x11] lapic_id[0x42] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x12] lapic_id[0x44] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x13] lapic_id[0x46] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x14] lapic_id[0x48] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x15] lapic_id[0x4a] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x16] lapic_id[0x4c] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x17] lapic_id[0x4e] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x18] lapic_id[0x60] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x19] lapic_id[0x62] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x64] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x66] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x68] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x6a] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x6c] enabled) :[ 0.000000] ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x6e] enabled) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x08] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x09] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x0a] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x0b] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x0c] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x0d] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x0e] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x0f] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x10] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x11] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x12] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x13] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x14] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x15] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x16] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x17] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x18] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x19] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x1a] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x1b] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x1c] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x1d] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x1e] high edge lint[0x1]) :[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0x1f] high edge lint[0x1]) :[ 0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0]) :[ 0.000000] IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23 :[ 0.000000] ACPI: IOAPIC (id[0x02] address[0xfec01000] gsi_base[24]) :[ 0.000000] IOAPIC[1]: apic_id 2, version 32, address 0xfec01000, GSI 24-47 :[ 0.000000] ACPI: IOAPIC (id[0x03] address[0xfec40000] gsi_base[48]) :[ 0.000000] IOAPIC[2]: apic_id 3, version 32, address 0xfec40000, GSI 48-71 :[ 0.000000] ACPI: IOAPIC (id[0x04] address[0xfec80000] gsi_base[72]) :[ 0.000000] IOAPIC[3]: apic_id 4, version 32, address 0xfec80000, GSI 72-95 :[ 0.000000] ACPI: IOAPIC (id[0x05] address[0xfecc0000] gsi_base[96]) :[ 0.000000] IOAPIC[4]: apic_id 5, version 32, address 0xfecc0000, GSI 96-119 :[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) :[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) :[ 0.000000] ACPI: IRQ0 used by override. :[ 0.000000] ACPI: IRQ9 used by override. :[ 0.000000] Using ACPI (MADT) for SMP configuration information :[ 0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000 :[ 0.000000] smpboot: Allowing 32 CPUs, 0 hotplug CPUs :[ 0.000000] PM: Registered nosave memory: [mem 0x0009a000-0x0009afff] :[ 0.000000] PM: Registered nosave memory: [mem 0x0009b000-0x0009ffff] :[ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000dffff] :[ 0.000000] PM: Registered nosave memory: [mem 0x000e0000-0x000fffff] :[ 0.000000] PM: Registered nosave memory: [mem 0x7df8c000-0x7e01afff] :[ 0.000000] PM: Registered nosave memory: [mem 0x7e01b000-0x7e36dfff] :[ 0.000000] PM: Registered nosave memory: [mem 0x7e36e000-0x7f35ffff] :[ 0.000000] PM: Registered nosave memory: [mem 0x7f360000-0x7f7fffff] :[ 0.000000] PM: Registered nosave memory: [mem 0x7f800000-0x7fffffff] :[ 0.000000] PM: Registered nosave memory: [mem 0x80000000-0x8fffffff] :[ 0.000000] PM: Registered nosave memory: [mem 0x90000000-0xfed1bfff] :[ 0.000000] PM: Registered nosave memory: [mem 0xfed1c000-0xfed3ffff] :[ 0.000000] PM: Registered nosave memory: [mem 0xfed40000-0xfeffffff] :[ 0.000000] PM: Registered nosave memory: [mem 0xff000000-0xffffffff] :[ 0.000000] e820: [mem 0x90000000-0xfed1bfff] available for PCI devices :[ 0.000000] Booting paravirtualized kernel on bare hardware :[ 0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:32 nr_cpu_ids:32 nr_node_ids:4 :[ 0.000000] PERCPU: Embedded 35 pages/cpu @ffff93a8ffc00000 s104920 r8192 d30248 u262144 :[ 0.000000] pcpu-alloc: s104920 r8192 d30248 u262144 alloc=1*2097152 :[ 0.000000] pcpu-alloc: [0] 00 01 02 03 04 05 06 07 [1] 08 09 10 11 12 13 14 15 :[ 0.000000] pcpu-alloc: [2] 16 17 18 19 20 21 22 23 [3] 24 25 26 27 28 29 30 31 :[ 0.000000] Built 4 zonelists in Zone order, mobility grouping on. Total pages: 132112273 :[ 0.000000] Policy zone: Normal :[ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.10.0-862.11.6.el7.x86_64 root=UUID=f47329d6-3b9a-419b-b11a-723d45086e59 ro crashkernel=auto rd.md.uuid=405aeaf1:88568fd8:ec8a357f:35f0e42f rd.md.uuid=d587da04:7ad06ea3:cd3e3baa:c7655e89 rhgb quiet LANG=en_US.UTF-8 :[ 0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes) :[ 0.000000] x86/fpu: xstate_offset[2]: 0240, xstate_sizes[2]: 0100 :[ 0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form :[ 0.000000] Memory: 10170732k/538968064k available (7348k kernel code, 2130796k absent, 8695088k reserved, 6281k data, 1836k init) :[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=32, Nodes=4 :[ 0.000000] x86/pti: Unmapping kernel while in userspace :[ 0.000000] Hierarchical RCU implementation. :[ 0.000000] RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=32. :[ 0.000000] NR_IRQS:327936 nr_irqs:2312 0 :[ 0.000000] Console: colour VGA+ 80x25 :[ 0.000000] console [tty0] enabled :[ 0.000000] allocated 2147483648 bytes of page_cgroup :[ 0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups :[ 0.000000] Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl :[ 0.000000] hpet clockevent registered :[ 0.000000] tsc: Fast TSC calibration using PIT :[ 0.000000] tsc: Detected 3299.952 MHz processor :[ 0.000053] Calibrating delay loop (skipped), value calculated using timer frequency.. 6599.90 BogoMIPS (lpj=3299952) :[ 0.000057] pid_max: default: 32768 minimum: 301 :[ 0.000196] Security Framework initialized :[ 0.000206] SELinux: Initializing. :[ 0.000219] SELinux: Starting in permissive mode :[ 0.000220] Yama: becoming mindful. :[ 0.032387] Dentry cache hash table entries: 67108864 (order: 17, 536870912 bytes) :[ 0.161752] Inode-cache hash table entries: 33554432 (order: 16, 268435456 bytes) :[ 0.217347] Mount-cache hash table entries: 1048576 (order: 11, 8388608 bytes) :[ 0.217832] Mountpoint-cache hash table entries: 1048576 (order: 11, 8388608 bytes) :[ 0.221634] Initializing cgroup subsys memory :[ 0.221664] Initializing cgroup subsys devices :[ 0.221666] Initializing cgroup subsys freezer :[ 0.221669] Initializing cgroup subsys net_cls :[ 0.221671] Initializing cgroup subsys blkio :[ 0.221672] Initializing cgroup subsys perf_event :[ 0.221675] Initializing cgroup subsys hugetlb :[ 0.221676] Initializing cgroup subsys pids :[ 0.221678] Initializing cgroup subsys net_prio :[ 0.222442] mce: CPU supports 25 MCE banks :[ 0.222463] CPU0: Thermal monitoring enabled (TM1) :[ 0.222509] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0 :[ 0.222510] Last level dTLB entries: 4KB 512, 2MB 0, 4MB 0 :[ 0.222511] tlb_flushall_shift: 6 :[ 0.222536] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl and seccomp :[ 0.222611] FEATURE SPEC_CTRL Present :[ 0.222612] FEATURE IBPB_SUPPORT Present :[ 0.222731] Spectre V2 : Mitigation: Full retpoline :[ 0.223669] Freeing SMP alternatives: 24k freed :[ 0.226201] ACPI: Core revision 20130517 :[ 0.248241] ACPI: All ACPI Tables successfully acquired :[ 0.248532] ftrace: allocating 28097 entries in 110 pages :[ 0.253976] random: fast init done :[ 0.270981] DMAR: Host address width 46 :[ 0.270983] DMAR: DRHD base: 0x000000fbffe000 flags: 0x0 :[ 0.270990] DMAR: dmar0: reg_base_addr fbffe000 ver 1:0 cap d2078c106f0466 ecap f020df :[ 0.270992] DMAR: DRHD base: 0x000000d3ffe000 flags: 0x0 :[ 0.270998] DMAR: dmar1: reg_base_addr d3ffe000 ver 1:0 cap d2078c106f0466 ecap f020df :[ 0.271000] DMAR: DRHD base: 0x000000cbffe000 flags: 0x0 :[ 0.271004] DMAR: dmar2: reg_base_addr cbffe000 ver 1:0 cap d2078c106f0466 ecap f020df :[ 0.271006] DMAR: DRHD base: 0x000000bbffc000 flags: 0x1 :[ 0.271010] DMAR: dmar3: reg_base_addr bbffc000 ver 1:0 cap d2078c106f0466 ecap f020df :[ 0.271011] DMAR: RMRR base: 0x0000007dff1000 end: 0x0000007dffdfff :[ 0.271013] DMAR: ATSR flags: 0x0 :[ 0.271014] DMAR: RHSA base: 0x000000fbffe000 proximity domain: 0x3 :[ 0.271016] DMAR: RHSA base: 0x000000d3ffe000 proximity domain: 0x2 :[ 0.271017] DMAR: RHSA base: 0x000000cbffe000 proximity domain: 0x1 :[ 0.271018] DMAR: RHSA base: 0x000000bbffc000 proximity domain: 0x0 :[ 0.271021] DMAR-IR: IOAPIC id 3 under DRHD base 0xcbffe000 IOMMU 2 :[ 0.271022] DMAR-IR: IOAPIC id 4 under DRHD base 0xd3ffe000 IOMMU 1 :[ 0.271023] DMAR-IR: IOAPIC id 5 under DRHD base 0xfbffe000 IOMMU 0 :[ 0.271024] DMAR-IR: IOAPIC id 0 under DRHD base 0xbbffc000 IOMMU 3 :[ 0.271025] DMAR-IR: IOAPIC id 2 under DRHD base 0xbbffc000 IOMMU 3 :[ 0.271027] DMAR-IR: HPET id 0 under DRHD base 0xbbffc000 :[ 0.271028] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping. :[ 0.273351] DMAR-IR: Enabled IRQ remapping in x2apic mode :[ 0.273353] Enabling x2apic :[ 0.273353] Enabled x2apic :[ 0.273361] Switched APIC routing to cluster x2apic. :[ 0.274346] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 :[ 0.284351] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-4627 v2 @ 3.30GHz (fam: 06, model: 3e, stepping: 04) :[ 0.284361] TSC deadline timer enabled :[ 0.284376] Performance Events: PEBS fmt1+, 16-deep LBR, IvyBridge events, full-width counters, Intel PMU driver. :[ 0.284398] ... version: 3 :[ 0.284399] ... bit width: 48 :[ 0.284399] ... generic registers: 8 :[ 0.284400] ... value mask: 0000ffffffffffff :[ 0.284401] ... max period: 00007fffffffffff :[ 0.284402] ... fixed-purpose events: 3 :[ 0.284403] ... event mask: 00000007000000ff :[ 0.294313] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter. :[ 0.286392] smpboot: Booting Node 0, Processors #1 #2 #3 #4 #5 #6 #7 OK :[ 0.321408] smpboot: Booting Node 1, Processors #8 #9 #10 #11 #12 #13 #14 #15 OK :[ 0.434627] smpboot: Booting Node 2, Processors #16 #17 #18 #19 #20 #21 #22 #23 OK :[ 0.549797] smpboot: Booting Node 3, Processors #24 #25 #26 #27 #28 #29 #30 #31 OK :[ 0.668103] Brought up 32 CPUs :[ 0.668106] smpboot: Max logical packages: 4 :[ 0.668111] smpboot: Total of 32 processors activated (211746.99 BogoMIPS) :[ 1.287297] node 0 initialised, 31981155 pages in 578ms :[ 1.297408] node 1 initialised, 32505494 pages in 588ms :[ 1.297987] node 2 initialised, 32505494 pages in 589ms :[ 1.298594] node 3 initialised, 32500719 pages in 589ms :[ 1.299131] devtmpfs: initialized :[ 1.350742] EVM: security.selinux :[ 1.350745] EVM: security.ima :[ 1.350746] EVM: security.capability :[ 1.350884] PM: Registering ACPI NVS region [mem 0x7e01b000-0x7e36dfff] (3485696 bytes) :[ 1.350964] PM: Registering ACPI NVS region [mem 0x7f360000-0x7f7fffff] (4849664 bytes) :[ 1.352426] atomic64 test passed for x86-64 platform with CX8 and with SSE :[ 1.352429] pinctrl core: initialized pinctrl subsystem :[ 1.352512] RTC time: 19:35:40, date: 08/28/18 :[ 1.352688] NET: Registered protocol family 16 :[ 1.352924] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it :[ 1.352927] ACPI: bus type PCI registered :[ 1.352929] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 :[ 1.353034] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0x80000000-0x8fffffff] (base 0x80000000) :[ 1.353037] PCI: MMCONFIG at [mem 0x80000000-0x8fffffff] reserved in E820 :[ 1.353058] pmd_set_huge: Cannot satisfy [mem 0x80000000-0x80200000] with a huge-page mapping due to MTRR override. :[ 1.353611] PCI: Using configuration type 1 for base access :[ 1.355958] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter. :[ 1.355999] core: PMU erratum BJ122, BV98, HSD29 workaround disabled, HT off :[ 1.374506] ACPI: Added _OSI(Module Device) :[ 1.374508] ACPI: Added _OSI(Processor Device) :[ 1.374509] ACPI: Added _OSI(3.0 _SCP Extensions) :[ 1.374510] ACPI: Added _OSI(Processor Aggregator Device) :[ 1.374511] ACPI: Added _OSI(Linux-Dell-Video) :[ 1.379922] ACPI: EC: Look up EC in DSDT :[ 1.385272] ACPI: Executed 1 blocks of module-level executable AML code :[ 1.541822] ACPI: Interpreter enabled :[ 1.541838] ACPI: (supports S0 S4 S5) :[ 1.541840] ACPI: Using IOAPIC for interrupt routing :[ 1.541911] HEST: Table parsing has been initialized. :[ 1.541915] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug :[ 1.541956] ACPI: GPE 0x10 active on init :[ 1.541970] ACPI: GPE 0x1B active on init :[ 1.541978] ACPI: Enabled 7 GPEs in block 00 to 3F :[ 1.542786] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored :[ 1.554376] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-3e]) :[ 1.554384] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] :[ 1.554484] acpi PNP0A08:00: _OSC: platform does not support [AER] :[ 1.554576] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] :[ 1.554577] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration :[ 1.554773] PCI host bridge to bus 0000:00 :[ 1.554777] pci_bus 0000:00: root bus resource [bus 00-3e] :[ 1.554779] pci_bus 0000:00: root bus resource [io 0x0000-0x03af window] :[ 1.554781] pci_bus 0000:00: root bus resource [io 0x03e0-0x0cf7 window] :[ 1.554783] pci_bus 0000:00: root bus resource [io 0x03b0-0x03df window] :[ 1.554784] pci_bus 0000:00: root bus resource [io 0x0d00-0x5fff window] :[ 1.554789] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window] :[ 1.554791] pci_bus 0000:00: root bus resource [mem 0x000c0000-0x000dffff window] :[ 1.554793] pci_bus 0000:00: root bus resource [mem 0xfed0e000-0xfed0ffff window] :[ 1.554794] pci_bus 0000:00: root bus resource [mem 0x80000000-0xbbffffff window] :[ 1.554808] pci 0000:00:00.0: [8086:0e00] type 00 class 0x060000 :[ 1.554887] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold :[ 1.554964] pci 0000:00:02.0: [8086:0e04] type 01 class 0x060400 :[ 1.555037] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold :[ 1.555079] pci 0000:00:02.0: System wakeup disabled by ACPI :[ 1.555125] pci 0000:00:03.0: [8086:0e08] type 01 class 0x060400 :[ 1.555199] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold :[ 1.555241] pci 0000:00:03.0: System wakeup disabled by ACPI :[ 1.555280] pci 0000:00:04.0: [8086:0e20] type 00 class 0x088000 :[ 1.555296] pci 0000:00:04.0: reg 0x10: [mem 0xbbb1c000-0xbbb1ffff 64bit] :[ 1.555405] pci 0000:00:04.1: [8086:0e21] type 00 class 0x088000 :[ 1.555420] pci 0000:00:04.1: reg 0x10: [mem 0xbbb18000-0xbbb1bfff 64bit] :[ 1.555524] pci 0000:00:04.2: [8086:0e22] type 00 class 0x088000 :[ 1.555538] pci 0000:00:04.2: reg 0x10: [mem 0xbbb14000-0xbbb17fff 64bit] :[ 1.555649] pci 0000:00:04.3: [8086:0e23] type 00 class 0x088000 :[ 1.555663] pci 0000:00:04.3: reg 0x10: [mem 0xbbb10000-0xbbb13fff 64bit] :[ 1.555772] pci 0000:00:04.4: [8086:0e24] type 00 class 0x088000 :[ 1.555786] pci 0000:00:04.4: reg 0x10: [mem 0xbbb0c000-0xbbb0ffff 64bit] :[ 1.555893] pci 0000:00:04.5: [8086:0e25] type 00 class 0x088000 :[ 1.555907] pci 0000:00:04.5: reg 0x10: [mem 0xbbb08000-0xbbb0bfff 64bit] :[ 1.556018] pci 0000:00:04.6: [8086:0e26] type 00 class 0x088000 :[ 1.556032] pci 0000:00:04.6: reg 0x10: [mem 0xbbb04000-0xbbb07fff 64bit] :[ 1.556142] pci 0000:00:04.7: [8086:0e27] type 00 class 0x088000 :[ 1.556156] pci 0000:00:04.7: reg 0x10: [mem 0xbbb00000-0xbbb03fff 64bit] :[ 1.556266] pci 0000:00:05.0: [8086:0e28] type 00 class 0x088000 :[ 1.556370] pci 0000:00:05.2: [8086:0e2a] type 00 class 0x088000 :[ 1.556477] pci 0000:00:05.4: [8086:0e2c] type 00 class 0x080020 :[ 1.556487] pci 0000:00:05.4: reg 0x10: [mem 0xbbb26000-0xbbb26fff] :[ 1.556604] pci 0000:00:11.0: [8086:1d3e] type 01 class 0x060400 :[ 1.556689] pci 0000:00:11.0: PME# supported from D0 D3hot D3cold :[ 1.556764] pci 0000:00:16.0: [8086:1d3a] type 00 class 0x078000 :[ 1.556781] pci 0000:00:16.0: reg 0x10: [mem 0xfed0e000-0xfed0e00f 64bit] :[ 1.556839] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold :[ 1.556906] pci 0000:00:16.1: [8086:1d3b] type 00 class 0x078000 :[ 1.556923] pci 0000:00:16.1: reg 0x10: [mem 0xfed0f000-0xfed0f00f 64bit] :[ 1.556980] pci 0000:00:16.1: PME# supported from D0 D3hot D3cold :[ 1.557058] pci 0000:00:1a.0: [8086:1d2d] type 00 class 0x0c0320 :[ 1.557074] pci 0000:00:1a.0: reg 0x10: [mem 0xbbb24000-0xbbb243ff] :[ 1.557151] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold :[ 1.557180] pci 0000:00:1a.0: System wakeup disabled by ACPI :[ 1.557224] pci 0000:00:1d.0: [8086:1d26] type 00 class 0x0c0320 :[ 1.557240] pci 0000:00:1d.0: reg 0x10: [mem 0xbbb23000-0xbbb233ff] :[ 1.557317] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold :[ 1.557349] pci 0000:00:1d.0: System wakeup disabled by ACPI :[ 1.557393] pci 0000:00:1e.0: [8086:244e] type 01 class 0x060401 :[ 1.557459] pci 0000:00:1e.0: System wakeup disabled by ACPI :[ 1.557497] pci 0000:00:1f.0: [8086:1d41] type 00 class 0x060100 :[ 1.557650] pci 0000:00:1f.2: [8086:1d02] type 00 class 0x010601 :[ 1.557664] pci 0000:00:1f.2: reg 0x10: [io 0x5070-0x5077] :[ 1.557671] pci 0000:00:1f.2: reg 0x14: [io 0x5060-0x5063] :[ 1.557677] pci 0000:00:1f.2: reg 0x18: [io 0x5050-0x5057] :[ 1.557684] pci 0000:00:1f.2: reg 0x1c: [io 0x5040-0x5043] :[ 1.557691] pci 0000:00:1f.2: reg 0x20: [io 0x5020-0x503f] :[ 1.557698] pci 0000:00:1f.2: reg 0x24: [mem 0xbbb22000-0xbbb227ff] :[ 1.557735] pci 0000:00:1f.2: PME# supported from D3hot :[ 1.557787] pci 0000:00:1f.3: [8086:1d22] type 00 class 0x0c0500 :[ 1.557801] pci 0000:00:1f.3: reg 0x10: [mem 0xbbb21000-0xbbb210ff 64bit] :[ 1.557819] pci 0000:00:1f.3: reg 0x20: [io 0x5000-0x501f] :[ 1.557896] pci 0000:00:1f.6: [8086:1d24] type 00 class 0x118000 :[ 1.557915] pci 0000:00:1f.6: reg 0x10: [mem 0xbbb20000-0xbbb20fff 64bit] :[ 1.558066] pci 0000:01:00.0: [1000:0097] type 00 class 0x010700 :[ 1.558075] pci 0000:01:00.0: reg 0x10: [io 0x4000-0x40ff] :[ 1.558082] pci 0000:01:00.0: reg 0x14: [mem 0xbba40000-0xbba4ffff 64bit] :[ 1.558089] pci 0000:01:00.0: reg 0x1c: [mem 0xbba00000-0xbba3ffff 64bit] :[ 1.558097] pci 0000:01:00.0: reg 0x30: [mem 0xbb900000-0xbb9fffff pref] :[ 1.558138] pci 0000:01:00.0: supports D1 D2 :[ 1.558190] pci 0000:00:02.0: PCI bridge to [bus 01] :[ 1.558194] pci 0000:00:02.0: bridge window [io 0x4000-0x4fff] :[ 1.558197] pci 0000:00:02.0: bridge window [mem 0xbb900000-0xbbafffff] :[ 1.558234] pci 0000:00:03.0: PCI bridge to [bus 02] :[ 1.558288] pci 0000:00:11.0: PCI bridge to [bus 03] :[ 1.558334] pci 0000:04:01.0: [102b:0532] type 00 class 0x030000 :[ 1.558349] pci 0000:04:01.0: reg 0x10: [mem 0xba000000-0xbaffffff pref] :[ 1.558357] pci 0000:04:01.0: reg 0x14: [mem 0xbb800000-0xbb803fff] :[ 1.558366] pci 0000:04:01.0: reg 0x18: [mem 0xbb000000-0xbb7fffff] :[ 1.558492] pci 0000:00:1e.0: PCI bridge to [bus 04] (subtractive decode) :[ 1.558497] pci 0000:00:1e.0: bridge window [mem 0xbb000000-0xbb8fffff] :[ 1.558501] pci 0000:00:1e.0: bridge window [mem 0xba000000-0xbaffffff 64bit pref] :[ 1.558503] pci 0000:00:1e.0: bridge window [io 0x0000-0x03af window] (subtractive decode) :[ 1.558505] pci 0000:00:1e.0: bridge window [io 0x03e0-0x0cf7 window] (subtractive decode) :[ 1.558507] pci 0000:00:1e.0: bridge window [io 0x03b0-0x03df window] (subtractive decode) :[ 1.558508] pci 0000:00:1e.0: bridge window [io 0x0d00-0x5fff window] (subtractive decode) :[ 1.558510] pci 0000:00:1e.0: bridge window [mem 0x000a0000-0x000bffff window] (subtractive decode) :[ 1.558511] pci 0000:00:1e.0: bridge window [mem 0x000c0000-0x000dffff window] (subtractive decode) :[ 1.558513] pci 0000:00:1e.0: bridge window [mem 0xfed0e000-0xfed0ffff window] (subtractive decode) :[ 1.558514] pci 0000:00:1e.0: bridge window [mem 0x80000000-0xbbffffff window] (subtractive decode) :[ 1.558537] pci_bus 0000:00: on NUMA node 0 :[ 1.558910] ACPI: PCI Root Bridge [UNC0] (domain 0000 [bus 3f]) :[ 1.558914] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] :[ 1.558930] acpi PNP0A03:00: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability] :[ 1.558931] acpi PNP0A03:00: FADT indicates ASPM is unsupported, using BIOS configuration :[ 1.558991] PCI host bridge to bus 0000:3f :[ 1.558993] pci_bus 0000:3f: root bus resource [bus 3f] :[ 1.559000] pci 0000:3f:08.0: [8086:0e80] type 00 class 0x088000 :[ 1.559052] pci 0000:3f:09.0: [8086:0e90] type 00 class 0x088000 :[ 1.559101] pci 0000:3f:0a.0: [8086:0ec0] type 00 class 0x088000 :[ 1.559146] pci 0000:3f:0a.1: [8086:0ec1] type 00 class 0x088000 :[ 1.559191] pci 0000:3f:0a.2: [8086:0ec2] type 00 class 0x088000 :[ 1.559233] pci 0000:3f:0a.3: [8086:0ec3] type 00 class 0x088000 :[ 1.559276] pci 0000:3f:0b.0: [8086:0e1e] type 00 class 0x088000 :[ 1.559322] pci 0000:3f:0b.3: [8086:0e1f] type 00 class 0x088000 :[ 1.559364] pci 0000:3f:0c.0: [8086:0ee0] type 00 class 0x088000 :[ 1.559413] pci 0000:3f:0c.1: [8086:0ee2] type 00 class 0x088000 :[ 1.559455] pci 0000:3f:0c.2: [8086:0ee4] type 00 class 0x088000 :[ 1.559494] pci 0000:3f:0c.3: [8086:0ee6] type 00 class 0x088000 :[ 1.559535] pci 0000:3f:0d.0: [8086:0ee1] type 00 class 0x088000 :[ 1.559573] pci 0000:3f:0d.1: [8086:0ee3] type 00 class 0x088000 :[ 1.559612] pci 0000:3f:0d.2: [8086:0ee5] type 00 class 0x088000 :[ 1.559653] pci 0000:3f:0d.3: [8086:0ee7] type 00 class 0x088000 :[ 1.559697] pci 0000:3f:0e.0: [8086:0ea0] type 00 class 0x088000 :[ 1.559740] pci 0000:3f:0e.1: [8086:0e30] type 00 class 0x110100 :[ 1.559791] pci 0000:3f:0f.0: [8086:0ea8] type 00 class 0x088000 :[ 1.559855] pci 0000:3f:0f.1: [8086:0e71] type 00 class 0x088000 :[ 1.559917] pci 0000:3f:0f.2: [8086:0eaa] type 00 class 0x088000 :[ 1.559977] pci 0000:3f:0f.3: [8086:0eab] type 00 class 0x088000 :[ 1.560037] pci 0000:3f:0f.4: [8086:0eac] type 00 class 0x088000 :[ 1.560095] pci 0000:3f:0f.5: [8086:0ead] type 00 class 0x088000 :[ 1.560151] pci 0000:3f:10.0: [8086:0eb0] type 00 class 0x088000 :[ 1.560209] pci 0000:3f:10.1: [8086:0eb1] type 00 class 0x088000 :[ 1.560271] pci 0000:3f:10.2: [8086:0eb2] type 00 class 0x088000 :[ 1.560327] pci 0000:3f:10.3: [8086:0eb3] type 00 class 0x088000 :[ 1.560386] pci 0000:3f:10.4: [8086:0eb4] type 00 class 0x088000 :[ 1.560443] pci 0000:3f:10.5: [8086:0eb5] type 00 class 0x088000 :[ 1.560503] pci 0000:3f:10.6: [8086:0eb6] type 00 class 0x088000 :[ 1.560564] pci 0000:3f:10.7: [8086:0eb7] type 00 class 0x088000 :[ 1.560620] pci 0000:3f:13.0: [8086:0e1d] type 00 class 0x088000 :[ 1.560664] pci 0000:3f:13.1: [8086:0e34] type 00 class 0x110100 :[ 1.560708] pci 0000:3f:13.4: [8086:0e81] type 00 class 0x088000 :[ 1.560753] pci 0000:3f:13.5: [8086:0e36] type 00 class 0x110100 :[ 1.560799] pci 0000:3f:16.0: [8086:0ec8] type 00 class 0x088000 :[ 1.560840] pci 0000:3f:16.1: [8086:0ec9] type 00 class 0x088000 :[ 1.560885] pci 0000:3f:16.2: [8086:0eca] type 00 class 0x088000 :[ 1.560938] pci_bus 0000:3f: on NUMA node 0 :[ 1.561027] ACPI: PCI Root Bridge [PCI1] (domain 0000 [bus 40-7e]) :[ 1.561030] acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] :[ 1.561130] acpi PNP0A08:01: _OSC: platform does not support [AER] :[ 1.561222] acpi PNP0A08:01: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] :[ 1.561226] acpi PNP0A08:01: FADT indicates ASPM is unsupported, using BIOS configuration :[ 1.561361] PCI host bridge to bus 0000:40 :[ 1.561363] pci_bus 0000:40: root bus resource [bus 40-7e] :[ 1.561364] pci_bus 0000:40: root bus resource [io 0x6000-0x7fff window] :[ 1.561366] pci_bus 0000:40: root bus resource [mem 0xbc000000-0xcbffffff window] :[ 1.561378] pci 0000:40:01.0: [8086:0e02] type 01 class 0x060400 :[ 1.561462] pci 0000:40:01.0: PME# supported from D0 D3hot D3cold :[ 1.561551] pci 0000:40:04.0: [8086:0e20] type 00 class 0x088000 :[ 1.561567] pci 0000:40:04.0: reg 0x10: [mem 0xcbf1c000-0xcbf1ffff 64bit] :[ 1.561679] pci 0000:40:04.1: [8086:0e21] type 00 class 0x088000 :[ 1.561695] pci 0000:40:04.1: reg 0x10: [mem 0xcbf18000-0xcbf1bfff 64bit] :[ 1.561812] pci 0000:40:04.2: [8086:0e22] type 00 class 0x088000 :[ 1.561828] pci 0000:40:04.2: reg 0x10: [mem 0xcbf14000-0xcbf17fff 64bit] :[ 1.561948] pci 0000:40:04.3: [8086:0e23] type 00 class 0x088000 :[ 1.561964] pci 0000:40:04.3: reg 0x10: [mem 0xcbf10000-0xcbf13fff 64bit] :[ 1.562080] pci 0000:40:04.4: [8086:0e24] type 00 class 0x088000 :[ 1.562096] pci 0000:40:04.4: reg 0x10: [mem 0xcbf0c000-0xcbf0ffff 64bit] :[ 1.562204] pci 0000:40:04.5: [8086:0e25] type 00 class 0x088000 :[ 1.562220] pci 0000:40:04.5: reg 0x10: [mem 0xcbf08000-0xcbf0bfff 64bit] :[ 1.562334] pci 0000:40:04.6: [8086:0e26] type 00 class 0x088000 :[ 1.562350] pci 0000:40:04.6: reg 0x10: [mem 0xcbf04000-0xcbf07fff 64bit] :[ 1.562470] pci 0000:40:04.7: [8086:0e27] type 00 class 0x088000 :[ 1.562485] pci 0000:40:04.7: reg 0x10: [mem 0xcbf00000-0xcbf03fff 64bit] :[ 1.562600] pci 0000:40:05.0: [8086:0e28] type 00 class 0x088000 :[ 1.562707] pci 0000:40:05.2: [8086:0e2a] type 00 class 0x088000 :[ 1.562814] pci 0000:40:05.4: [8086:0e2c] type 00 class 0x080020 :[ 1.562826] pci 0000:40:05.4: reg 0x10: [mem 0xcbf20000-0xcbf20fff] :[ 1.562984] pci 0000:40:01.0: PCI bridge to [bus 41] :[ 1.563000] pci_bus 0000:40: on NUMA node 1 :[ 1.563043] ACPI: PCI Root Bridge [UNC1] (domain 0000 [bus 7f]) :[ 1.563048] acpi PNP0A03:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] :[ 1.563064] acpi PNP0A03:01: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability] :[ 1.563066] acpi PNP0A03:01: FADT indicates ASPM is unsupported, using BIOS configuration :[ 1.563113] PCI host bridge to bus 0000:7f :[ 1.563115] pci_bus 0000:7f: root bus resource [bus 7f] :[ 1.563123] pci 0000:7f:08.0: [8086:0e80] type 00 class 0x088000 :[ 1.563187] pci 0000:7f:09.0: [8086:0e90] type 00 class 0x088000 :[ 1.563242] pci 0000:7f:0a.0: [8086:0ec0] type 00 class 0x088000 :[ 1.563296] pci 0000:7f:0a.1: [8086:0ec1] type 00 class 0x088000 :[ 1.563349] pci 0000:7f:0a.2: [8086:0ec2] type 00 class 0x088000 :[ 1.563398] pci 0000:7f:0a.3: [8086:0ec3] type 00 class 0x088000 :[ 1.563451] pci 0000:7f:0b.0: [8086:0e1e] type 00 class 0x088000 :[ 1.563503] pci 0000:7f:0b.3: [8086:0e1f] type 00 class 0x088000 :[ 1.563553] pci 0000:7f:0c.0: [8086:0ee0] type 00 class 0x088000 :[ 1.563606] pci 0000:7f:0c.1: [8086:0ee2] type 00 class 0x088000 :[ 1.563653] pci 0000:7f:0c.2: [8086:0ee4] type 00 class 0x088000 :[ 1.563704] pci 0000:7f:0c.3: [8086:0ee6] type 00 class 0x088000 :[ 1.563760] pci 0000:7f:0d.0: [8086:0ee1] type 00 class 0x088000 :[ 1.563810] pci 0000:7f:0d.1: [8086:0ee3] type 00 class 0x088000 :[ 1.563860] pci 0000:7f:0d.2: [8086:0ee5] type 00 class 0x088000 :[ 1.563911] pci 0000:7f:0d.3: [8086:0ee7] type 00 class 0x088000 :[ 1.563963] pci 0000:7f:0e.0: [8086:0ea0] type 00 class 0x088000 :[ 1.564021] pci 0000:7f:0e.1: [8086:0e30] type 00 class 0x110100 :[ 1.564082] pci 0000:7f:0f.0: [8086:0ea8] type 00 class 0x088000 :[ 1.564155] pci 0000:7f:0f.1: [8086:0e71] type 00 class 0x088000 :[ 1.564225] pci 0000:7f:0f.2: [8086:0eaa] type 00 class 0x088000 :[ 1.564289] pci 0000:7f:0f.3: [8086:0eab] type 00 class 0x088000 :[ 1.564363] pci 0000:7f:0f.4: [8086:0eac] type 00 class 0x088000 :[ 1.564426] pci 0000:7f:0f.5: [8086:0ead] type 00 class 0x088000 :[ 1.564500] pci 0000:7f:10.0: [8086:0eb0] type 00 class 0x088000 :[ 1.564565] pci 0000:7f:10.1: [8086:0eb1] type 00 class 0x088000 :[ 1.564636] pci 0000:7f:10.2: [8086:0eb2] type 00 class 0x088000 :[ 1.564703] pci 0000:7f:10.3: [8086:0eb3] type 00 class 0x088000 :[ 1.564789] pci 0000:7f:10.4: [8086:0eb4] type 00 class 0x088000 :[ 1.564857] pci 0000:7f:10.5: [8086:0eb5] type 00 class 0x088000 :[ 1.564926] pci 0000:7f:10.6: [8086:0eb6] type 00 class 0x088000 :[ 1.565001] pci 0000:7f:10.7: [8086:0eb7] type 00 class 0x088000 :[ 1.565074] pci 0000:7f:13.0: [8086:0e1d] type 00 class 0x088000 :[ 1.565125] pci 0000:7f:13.1: [8086:0e34] type 00 class 0x110100 :[ 1.565173] pci 0000:7f:13.4: [8086:0e81] type 00 class 0x088000 :[ 1.565227] pci 0000:7f:13.5: [8086:0e36] type 00 class 0x110100 :[ 1.565284] pci 0000:7f:16.0: [8086:0ec8] type 00 class 0x088000 :[ 1.565337] pci 0000:7f:16.1: [8086:0ec9] type 00 class 0x088000 :[ 1.565385] pci 0000:7f:16.2: [8086:0eca] type 00 class 0x088000 :[ 1.565442] pci_bus 0000:7f: on NUMA node 1 :[ 1.565521] ACPI: PCI Root Bridge [PCI2] (domain 0000 [bus 80-be]) :[ 1.565524] acpi PNP0A08:02: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] :[ 1.565619] acpi PNP0A08:02: _OSC: platform does not support [AER] :[ 1.565706] acpi PNP0A08:02: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] :[ 1.565708] acpi PNP0A08:02: FADT indicates ASPM is unsupported, using BIOS configuration :[ 1.565838] PCI host bridge to bus 0000:80 :[ 1.565840] pci_bus 0000:80: root bus resource [bus 80-be] :[ 1.565842] pci_bus 0000:80: root bus resource [io 0x8000-0x9fff window] :[ 1.565844] pci_bus 0000:80: root bus resource [mem 0xcc000000-0xd3ffffff window] :[ 1.565857] pci 0000:80:04.0: [8086:0e20] type 00 class 0x088000 :[ 1.565879] pci 0000:80:04.0: reg 0x10: [mem 0xd3f1c000-0xd3f1ffff 64bit] :[ 1.566004] pci 0000:80:04.1: [8086:0e21] type 00 class 0x088000 :[ 1.566021] pci 0000:80:04.1: reg 0x10: [mem 0xd3f18000-0xd3f1bfff 64bit] :[ 1.566136] pci 0000:80:04.2: [8086:0e22] type 00 class 0x088000 :[ 1.566153] pci 0000:80:04.2: reg 0x10: [mem 0xd3f14000-0xd3f17fff 64bit] :[ 1.566275] pci 0000:80:04.3: [8086:0e23] type 00 class 0x088000 :[ 1.566292] pci 0000:80:04.3: reg 0x10: [mem 0xd3f10000-0xd3f13fff 64bit] :[ 1.566407] pci 0000:80:04.4: [8086:0e24] type 00 class 0x088000 :[ 1.566423] pci 0000:80:04.4: reg 0x10: [mem 0xd3f0c000-0xd3f0ffff 64bit] :[ 1.566550] pci 0000:80:04.5: [8086:0e25] type 00 class 0x088000 :[ 1.566567] pci 0000:80:04.5: reg 0x10: [mem 0xd3f08000-0xd3f0bfff 64bit] :[ 1.566678] pci 0000:80:04.6: [8086:0e26] type 00 class 0x088000 :[ 1.566695] pci 0000:80:04.6: reg 0x10: [mem 0xd3f04000-0xd3f07fff 64bit] :[ 1.566815] pci 0000:80:04.7: [8086:0e27] type 00 class 0x088000 :[ 1.566832] pci 0000:80:04.7: reg 0x10: [mem 0xd3f00000-0xd3f03fff 64bit] :[ 1.566947] pci 0000:80:05.0: [8086:0e28] type 00 class 0x088000 :[ 1.567065] pci 0000:80:05.2: [8086:0e2a] type 00 class 0x088000 :[ 1.567170] pci 0000:80:05.4: [8086:0e2c] type 00 class 0x080020 :[ 1.567183] pci 0000:80:05.4: reg 0x10: [mem 0xd3f20000-0xd3f20fff] :[ 1.567310] pci_bus 0000:80: on NUMA node 2 :[ 1.567353] ACPI: PCI Root Bridge [UNC2] (domain 0000 [bus bf]) :[ 1.567356] acpi PNP0A03:02: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] :[ 1.567371] acpi PNP0A03:02: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability] :[ 1.567373] acpi PNP0A03:02: FADT indicates ASPM is unsupported, using BIOS configuration :[ 1.567421] PCI host bridge to bus 0000:bf :[ 1.567423] pci_bus 0000:bf: root bus resource [bus bf] :[ 1.567432] pci 0000:bf:08.0: [8086:0e80] type 00 class 0x088000 :[ 1.567499] pci 0000:bf:09.0: [8086:0e90] type 00 class 0x088000 :[ 1.567563] pci 0000:bf:0a.0: [8086:0ec0] type 00 class 0x088000 :[ 1.567616] pci 0000:bf:0a.1: [8086:0ec1] type 00 class 0x088000 :[ 1.567665] pci 0000:bf:0a.2: [8086:0ec2] type 00 class 0x088000 :[ 1.567717] pci 0000:bf:0a.3: [8086:0ec3] type 00 class 0x088000 :[ 1.567769] pci 0000:bf:0b.0: [8086:0e1e] type 00 class 0x088000 :[ 1.567819] pci 0000:bf:0b.3: [8086:0e1f] type 00 class 0x088000 :[ 1.567868] pci 0000:bf:0c.0: [8086:0ee0] type 00 class 0x088000 :[ 1.567922] pci 0000:bf:0c.1: [8086:0ee2] type 00 class 0x088000 :[ 1.567976] pci 0000:bf:0c.2: [8086:0ee4] type 00 class 0x088000 :[ 1.568026] pci 0000:bf:0c.3: [8086:0ee6] type 00 class 0x088000 :[ 1.568077] pci 0000:bf:0d.0: [8086:0ee1] type 00 class 0x088000 :[ 1.568129] pci 0000:bf:0d.1: [8086:0ee3] type 00 class 0x088000 :[ 1.568185] pci 0000:bf:0d.2: [8086:0ee5] type 00 class 0x088000 :[ 1.568239] pci 0000:bf:0d.3: [8086:0ee7] type 00 class 0x088000 :[ 1.568302] pci 0000:bf:0e.0: [8086:0ea0] type 00 class 0x088000 :[ 1.568369] pci 0000:bf:0e.1: [8086:0e30] type 00 class 0x110100 :[ 1.568438] pci 0000:bf:0f.0: [8086:0ea8] type 00 class 0x088000 :[ 1.568516] pci 0000:bf:0f.1: [8086:0e71] type 00 class 0x088000 :[ 1.568595] pci 0000:bf:0f.2: [8086:0eaa] type 00 class 0x088000 :[ 1.568668] pci 0000:bf:0f.3: [8086:0eab] type 00 class 0x088000 :[ 1.568746] pci 0000:bf:0f.4: [8086:0eac] type 00 class 0x088000 :[ 1.568825] pci 0000:bf:0f.5: [8086:0ead] type 00 class 0x088000 :[ 1.568904] pci 0000:bf:10.0: [8086:0eb0] type 00 class 0x088000 :[ 1.568979] pci 0000:bf:10.1: [8086:0eb1] type 00 class 0x088000 :[ 1.569060] pci 0000:bf:10.2: [8086:0eb2] type 00 class 0x088000 :[ 1.569139] pci 0000:bf:10.3: [8086:0eb3] type 00 class 0x088000 :[ 1.569217] pci 0000:bf:10.4: [8086:0eb4] type 00 class 0x088000 :[ 1.569298] pci 0000:bf:10.5: [8086:0eb5] type 00 class 0x088000 :[ 1.569376] pci 0000:bf:10.6: [8086:0eb6] type 00 class 0x088000 :[ 1.569455] pci 0000:bf:10.7: [8086:0eb7] type 00 class 0x088000 :[ 1.569533] pci 0000:bf:13.0: [8086:0e1d] type 00 class 0x088000 :[ 1.569589] pci 0000:bf:13.1: [8086:0e34] type 00 class 0x110100 :[ 1.569650] pci 0000:bf:13.4: [8086:0e81] type 00 class 0x088000 :[ 1.569707] pci 0000:bf:13.5: [8086:0e36] type 00 class 0x110100 :[ 1.569768] pci 0000:bf:16.0: [8086:0ec8] type 00 class 0x088000 :[ 1.569825] pci 0000:bf:16.1: [8086:0ec9] type 00 class 0x088000 :[ 1.569884] pci 0000:bf:16.2: [8086:0eca] type 00 class 0x088000 :[ 1.569949] pci_bus 0000:bf: on NUMA node 2 :[ 1.570028] ACPI: PCI Root Bridge [PCI3] (domain 0000 [bus c0-fe]) :[ 1.570031] acpi PNP0A08:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] :[ 1.570126] acpi PNP0A08:03: _OSC: platform does not support [AER] :[ 1.570213] acpi PNP0A08:03: _OSC: OS now controls [PCIeHotplug PME PCIeCapability] :[ 1.570215] acpi PNP0A08:03: FADT indicates ASPM is unsupported, using BIOS configuration :[ 1.570345] PCI host bridge to bus 0000:c0 :[ 1.570347] pci_bus 0000:c0: root bus resource [bus c0-fe] :[ 1.570349] pci_bus 0000:c0: root bus resource [io 0xa000-0xffff window] :[ 1.570350] pci_bus 0000:c0: root bus resource [mem 0xd4000000-0xfbffffff window] :[ 1.570363] pci 0000:c0:02.0: [8086:0e04] type 01 class 0x060400 :[ 1.570450] pci 0000:c0:02.0: PME# supported from D0 D3hot D3cold :[ 1.570490] pci 0000:c0:02.0: System wakeup disabled by ACPI :[ 1.570540] pci 0000:c0:02.2: [8086:0e06] type 01 class 0x060400 :[ 1.570623] pci 0000:c0:02.2: PME# supported from D0 D3hot D3cold :[ 1.570662] pci 0000:c0:02.2: System wakeup disabled by ACPI :[ 1.570706] pci 0000:c0:03.0: [8086:0e08] type 01 class 0x060400 :[ 1.570790] pci 0000:c0:03.0: PME# supported from D0 D3hot D3cold :[ 1.570829] pci 0000:c0:03.0: System wakeup disabled by ACPI :[ 1.570875] pci 0000:c0:04.0: [8086:0e20] type 00 class 0x088000 :[ 1.570894] pci 0000:c0:04.0: reg 0x10: [mem 0xfbf1c000-0xfbf1ffff 64bit] :[ 1.571011] pci 0000:c0:04.1: [8086:0e21] type 00 class 0x088000 :[ 1.571027] pci 0000:c0:04.1: reg 0x10: [mem 0xfbf18000-0xfbf1bfff 64bit] :[ 1.571142] pci 0000:c0:04.2: [8086:0e22] type 00 class 0x088000 :[ 1.571158] pci 0000:c0:04.2: reg 0x10: [mem 0xfbf14000-0xfbf17fff 64bit] :[ 1.571275] pci 0000:c0:04.3: [8086:0e23] type 00 class 0x088000 :[ 1.571290] pci 0000:c0:04.3: reg 0x10: [mem 0xfbf10000-0xfbf13fff 64bit] :[ 1.571405] pci 0000:c0:04.4: [8086:0e24] type 00 class 0x088000 :[ 1.571421] pci 0000:c0:04.4: reg 0x10: [mem 0xfbf0c000-0xfbf0ffff 64bit] :[ 1.571535] pci 0000:c0:04.5: [8086:0e25] type 00 class 0x088000 :[ 1.571551] pci 0000:c0:04.5: reg 0x10: [mem 0xfbf08000-0xfbf0bfff 64bit] :[ 1.571659] pci 0000:c0:04.6: [8086:0e26] type 00 class 0x088000 :[ 1.571674] pci 0000:c0:04.6: reg 0x10: [mem 0xfbf04000-0xfbf07fff 64bit] :[ 1.571790] pci 0000:c0:04.7: [8086:0e27] type 00 class 0x088000 :[ 1.571806] pci 0000:c0:04.7: reg 0x10: [mem 0xfbf00000-0xfbf03fff 64bit] :[ 1.571922] pci 0000:c0:05.0: [8086:0e28] type 00 class 0x088000 :[ 1.572029] pci 0000:c0:05.2: [8086:0e2a] type 00 class 0x088000 :[ 1.572135] pci 0000:c0:05.4: [8086:0e2c] type 00 class 0x080020 :[ 1.572146] pci 0000:c0:05.4: reg 0x10: [mem 0xfbf20000-0xfbf20fff] :[ 1.572322] pci 0000:c1:00.0: [8086:1521] type 00 class 0x020000 :[ 1.572339] pci 0000:c1:00.0: reg 0x10: [mem 0xfbe20000-0xfbe3ffff] :[ 1.572353] pci 0000:c1:00.0: reg 0x18: [io 0xf020-0xf03f] :[ 1.572361] pci 0000:c1:00.0: reg 0x1c: [mem 0xfbec4000-0xfbec7fff] :[ 1.572428] pci 0000:c1:00.0: PME# supported from D0 D3hot D3cold :[ 1.572454] pci 0000:c1:00.0: reg 0x184: [mem 0xfbea0000-0xfbea3fff] :[ 1.572456] pci 0000:c1:00.0: VF(n) BAR0 space: [mem 0xfbea0000-0xfbebffff] (contains BAR0 for 8 VFs) :[ 1.572473] pci 0000:c1:00.0: reg 0x190: [mem 0xfbe80000-0xfbe83fff] :[ 1.572475] pci 0000:c1:00.0: VF(n) BAR3 space: [mem 0xfbe80000-0xfbe9ffff] (contains BAR3 for 8 VFs) :[ 1.572571] pci 0000:c1:00.1: [8086:1521] type 00 class 0x020000 :[ 1.572587] pci 0000:c1:00.1: reg 0x10: [mem 0xfbe00000-0xfbe1ffff] :[ 1.572600] pci 0000:c1:00.1: reg 0x18: [io 0xf000-0xf01f] :[ 1.572607] pci 0000:c1:00.1: reg 0x1c: [mem 0xfbec0000-0xfbec3fff] :[ 1.572673] pci 0000:c1:00.1: PME# supported from D0 D3hot D3cold :[ 1.572694] pci 0000:c1:00.1: reg 0x184: [mem 0xfbe60000-0xfbe63fff] :[ 1.572695] pci 0000:c1:00.1: VF(n) BAR0 space: [mem 0xfbe60000-0xfbe7ffff] (contains BAR0 for 8 VFs) :[ 1.572714] pci 0000:c1:00.1: reg 0x190: [mem 0xfbe40000-0xfbe43fff] :[ 1.572716] pci 0000:c1:00.1: VF(n) BAR3 space: [mem 0xfbe40000-0xfbe5ffff] (contains BAR3 for 8 VFs) :[ 1.574920] pci 0000:c0:02.0: PCI bridge to [bus c1-c2] :[ 1.574939] pci 0000:c0:02.0: bridge window [io 0xf000-0xffff] :[ 1.574942] pci 0000:c0:02.0: bridge window [mem 0xfbe00000-0xfbefffff] :[ 1.574989] pci 0000:c0:02.2: PCI bridge to [bus c3] :[ 1.575043] pci 0000:c0:03.0: PCI bridge to [bus c4] :[ 1.575071] pci_bus 0000:c0: on NUMA node 3 :[ 1.575121] ACPI: PCI Root Bridge [UNC3] (domain 0000 [bus ff]) :[ 1.575125] acpi PNP0A03:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI] :[ 1.575140] acpi PNP0A03:03: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability] :[ 1.575142] acpi PNP0A03:03: FADT indicates ASPM is unsupported, using BIOS configuration :[ 1.575188] PCI host bridge to bus 0000:ff :[ 1.575190] pci_bus 0000:ff: root bus resource [bus ff] :[ 1.575198] pci 0000:ff:08.0: [8086:0e80] type 00 class 0x088000 :[ 1.575262] pci 0000:ff:09.0: [8086:0e90] type 00 class 0x088000 :[ 1.575319] pci 0000:ff:0a.0: [8086:0ec0] type 00 class 0x088000 :[ 1.575371] pci 0000:ff:0a.1: [8086:0ec1] type 00 class 0x088000 :[ 1.575419] pci 0000:ff:0a.2: [8086:0ec2] type 00 class 0x088000 :[ 1.575470] pci 0000:ff:0a.3: [8086:0ec3] type 00 class 0x088000 :[ 1.575528] pci 0000:ff:0b.0: [8086:0e1e] type 00 class 0x088000 :[ 1.575577] pci 0000:ff:0b.3: [8086:0e1f] type 00 class 0x088000 :[ 1.575627] pci 0000:ff:0c.0: [8086:0ee0] type 00 class 0x088000 :[ 1.575672] pci 0000:ff:0c.1: [8086:0ee2] type 00 class 0x088000 :[ 1.575724] pci 0000:ff:0c.2: [8086:0ee4] type 00 class 0x088000 :[ 1.575775] pci 0000:ff:0c.3: [8086:0ee6] type 00 class 0x088000 :[ 1.575826] pci 0000:ff:0d.0: [8086:0ee1] type 00 class 0x088000 :[ 1.575875] pci 0000:ff:0d.1: [8086:0ee3] type 00 class 0x088000 :[ 1.575926] pci 0000:ff:0d.2: [8086:0ee5] type 00 class 0x088000 :[ 1.575977] pci 0000:ff:0d.3: [8086:0ee7] type 00 class 0x088000 :[ 1.576024] pci 0000:ff:0e.0: [8086:0ea0] type 00 class 0x088000 :[ 1.576081] pci 0000:ff:0e.1: [8086:0e30] type 00 class 0x110100 :[ 1.576141] pci 0000:ff:0f.0: [8086:0ea8] type 00 class 0x088000 :[ 1.576215] pci 0000:ff:0f.1: [8086:0e71] type 00 class 0x088000 :[ 1.576286] pci 0000:ff:0f.2: [8086:0eaa] type 00 class 0x088000 :[ 1.576350] pci 0000:ff:0f.3: [8086:0eab] type 00 class 0x088000 :[ 1.576421] pci 0000:ff:0f.4: [8086:0eac] type 00 class 0x088000 :[ 1.576485] pci 0000:ff:0f.5: [8086:0ead] type 00 class 0x088000 :[ 1.576558] pci 0000:ff:10.0: [8086:0eb0] type 00 class 0x088000 :[ 1.576623] pci 0000:ff:10.1: [8086:0eb1] type 00 class 0x088000 :[ 1.576694] pci 0000:ff:10.2: [8086:0eb2] type 00 class 0x088000 :[ 1.576763] pci 0000:ff:10.3: [8086:0eb3] type 00 class 0x088000 :[ 1.576831] pci 0000:ff:10.4: [8086:0eb4] type 00 class 0x088000 :[ 1.576905] pci 0000:ff:10.5: [8086:0eb5] type 00 class 0x088000 :[ 1.576978] pci 0000:ff:10.6: [8086:0eb6] type 00 class 0x088000 :[ 1.577051] pci 0000:ff:10.7: [8086:0eb7] type 00 class 0x088000 :[ 1.577122] pci 0000:ff:13.0: [8086:0e1d] type 00 class 0x088000 :[ 1.577170] pci 0000:ff:13.1: [8086:0e34] type 00 class 0x110100 :[ 1.577220] pci 0000:ff:13.4: [8086:0e81] type 00 class 0x088000 :[ 1.577286] pci 0000:ff:13.5: [8086:0e36] type 00 class 0x110100 :[ 1.577341] pci 0000:ff:16.0: [8086:0ec8] type 00 class 0x088000 :[ 1.577389] pci 0000:ff:16.1: [8086:0ec9] type 00 class 0x088000 :[ 1.577442] pci 0000:ff:16.2: [8086:0eca] type 00 class 0x088000 :[ 1.577499] pci_bus 0000:ff: on NUMA node 3 :[ 1.577554] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15) :[ 1.577588] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 *10 11 12 14 15) :[ 1.577621] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 *5 6 10 11 12 14 15) :[ 1.577653] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 10 *11 12 14 15) :[ 1.577686] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 12 14 15) *0 :[ 1.577718] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0 :[ 1.577751] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 12 14 15) *0 :[ 1.577783] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 *7 10 11 12 14 15) :[ 1.581539] vgaarb: device added: PCI:0000:04:01.0,decodes=io+mem,owns=io+mem,locks=none :[ 1.581558] vgaarb: loaded :[ 1.581558] vgaarb: bridge control possible 0000:04:01.0 :[ 1.581724] SCSI subsystem initialized :[ 1.581756] ACPI: bus type USB registered :[ 1.581782] usbcore: registered new interface driver usbfs :[ 1.581790] usbcore: registered new interface driver hub :[ 1.581911] usbcore: registered new device driver usb :[ 1.582132] EDAC MC: Ver: 3.0.0 :[ 1.582598] PCI: Using ACPI for IRQ routing :[ 1.586008] PCI: pci_cache_line_size set to 64 bytes :[ 1.586285] e820: reserve RAM buffer [mem 0x0009ac00-0x0009ffff] :[ 1.586286] e820: reserve RAM buffer [mem 0x7df8c000-0x7fffffff] :[ 1.586474] NetLabel: Initializing :[ 1.586475] NetLabel: domain hash size = 128 :[ 1.586476] NetLabel: protocols = UNLABELED CIPSOv4 :[ 1.586506] NetLabel: unlabeled traffic allowed by default :[ 1.586560] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0 :[ 1.586564] hpet0: 8 comparators, 64-bit 14.318180 MHz counter :[ 1.588609] amd_nb: Cannot enumerate AMD northbridges :[ 1.588751] Switched to clocksource hpet :[ 1.596625] pnp: PnP ACPI init :[ 1.596652] ACPI: bus type PNP registered :[ 1.596944] system 00:00: [mem 0xfc000000-0xfcffffff] has been reserved :[ 1.596946] system 00:00: [mem 0xfd000000-0xfdffffff] has been reserved :[ 1.596948] system 00:00: [mem 0xfe000000-0xfeafffff] has been reserved :[ 1.596950] system 00:00: [mem 0xfeb00000-0xfebfffff] has been reserved :[ 1.596952] system 00:00: [mem 0xfed00400-0xfed3ffff] could not be reserved :[ 1.596954] system 00:00: [mem 0xfed45000-0xfedfffff] has been reserved :[ 1.596959] system 00:00: Plug and Play ACPI device, IDs PNP0c01 (active) :[ 1.597050] system 00:01: [mem 0xbbffc000-0xbbffdfff] could not be reserved :[ 1.597052] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active) :[ 1.597165] system 00:02: [io 0x0b00-0x0b7f] has been reserved :[ 1.597168] system 00:02: Plug and Play ACPI device, IDs PNP0c02 (active) :[ 1.597297] pnp 00:03: [dma 0 disabled] :[ 1.597345] pnp 00:03: Plug and Play ACPI device, IDs PNP0501 (active) :[ 1.597477] pnp 00:04: [dma 0 disabled] :[ 1.597521] pnp 00:04: Plug and Play ACPI device, IDs PNP0501 (active) :[ 1.597551] pnp 00:05: Plug and Play ACPI device, IDs PNP0b00 (active) :[ 1.597621] system 00:06: [io 0x04d0-0x04d1] has been reserved :[ 1.597623] system 00:06: [mem 0x00000400-0x000004ff] could not be reserved :[ 1.597625] system 00:06: Plug and Play ACPI device, IDs PNP0c02 (active) :[ 1.597815] system 00:07: [io 0x0400-0x0453] could not be reserved :[ 1.597817] system 00:07: [io 0x0458-0x047f] has been reserved :[ 1.597819] system 00:07: [io 0x1180-0x119f] has been reserved :[ 1.597820] system 00:07: [io 0x0500-0x057f] has been reserved :[ 1.597822] system 00:07: [mem 0xfed1c000-0xfed1ffff] has been reserved :[ 1.597824] system 00:07: [mem 0xfec00000-0xfecfffff] could not be reserved :[ 1.597826] system 00:07: [mem 0xfed08000-0xfed08fff] has been reserved :[ 1.597828] system 00:07: [mem 0xff000000-0xffffffff] has been reserved :[ 1.597830] system 00:07: Plug and Play ACPI device, IDs PNP0c01 (active) :[ 1.597896] system 00:08: [io 0x0454-0x0457] has been reserved :[ 1.597898] system 00:08: Plug and Play ACPI device, IDs INT3f0d PNP0c02 (active) :[ 1.598052] system 00:09: Plug and Play ACPI device, IDs PNP0c01 (active) :[ 1.598135] system 00:0a: [mem 0xcbffe000-0xcbffffff] could not be reserved :[ 1.598138] system 00:0a: Plug and Play ACPI device, IDs PNP0c02 (active) :[ 1.598208] system 00:0b: Plug and Play ACPI device, IDs PNP0c01 (active) :[ 1.598287] system 00:0c: [mem 0xd3ffe000-0xd3ffffff] could not be reserved :[ 1.598289] system 00:0c: Plug and Play ACPI device, IDs PNP0c02 (active) :[ 1.598360] system 00:0d: Plug and Play ACPI device, IDs PNP0c01 (active) :[ 1.598436] system 00:0e: [mem 0xfbffe000-0xfbffffff] could not be reserved :[ 1.598439] system 00:0e: Plug and Play ACPI device, IDs PNP0c02 (active) :[ 1.598836] pnp 00:0f: Plug and Play ACPI device, IDs PNP0c31 (active) :[ 1.598841] pnp: PnP ACPI: found 16 devices :[ 1.598842] ACPI: bus type PNP unregistered :[ 1.608082] pci 0000:00:02.0: PCI bridge to [bus 01] :[ 1.608086] pci 0000:00:02.0: bridge window [io 0x4000-0x4fff] :[ 1.608091] pci 0000:00:02.0: bridge window [mem 0xbb900000-0xbbafffff] :[ 1.608098] pci 0000:00:03.0: PCI bridge to [bus 02] :[ 1.608108] pci 0000:00:11.0: PCI bridge to [bus 03] :[ 1.608121] pci 0000:00:1e.0: PCI bridge to [bus 04] :[ 1.608125] pci 0000:00:1e.0: bridge window [mem 0xbb000000-0xbb8fffff] :[ 1.608128] pci 0000:00:1e.0: bridge window [mem 0xba000000-0xbaffffff 64bit pref] :[ 1.608135] pci_bus 0000:00: resource 4 [io 0x0000-0x03af window] :[ 1.608136] pci_bus 0000:00: resource 5 [io 0x03e0-0x0cf7 window] :[ 1.608138] pci_bus 0000:00: resource 6 [io 0x03b0-0x03df window] :[ 1.608140] pci_bus 0000:00: resource 7 [io 0x0d00-0x5fff window] :[ 1.608142] pci_bus 0000:00: resource 8 [mem 0x000a0000-0x000bffff window] :[ 1.608144] pci_bus 0000:00: resource 9 [mem 0x000c0000-0x000dffff window] :[ 1.608145] pci_bus 0000:00: resource 10 [mem 0xfed0e000-0xfed0ffff window] :[ 1.608147] pci_bus 0000:00: resource 11 [mem 0x80000000-0xbbffffff window] :[ 1.608149] pci_bus 0000:01: resource 0 [io 0x4000-0x4fff] :[ 1.608151] pci_bus 0000:01: resource 1 [mem 0xbb900000-0xbbafffff] :[ 1.608153] pci_bus 0000:04: resource 1 [mem 0xbb000000-0xbb8fffff] :[ 1.608155] pci_bus 0000:04: resource 2 [mem 0xba000000-0xbaffffff 64bit pref] :[ 1.608156] pci_bus 0000:04: resource 4 [io 0x0000-0x03af window] :[ 1.608158] pci_bus 0000:04: resource 5 [io 0x03e0-0x0cf7 window] :[ 1.608159] pci_bus 0000:04: resource 6 [io 0x03b0-0x03df window] :[ 1.608161] pci_bus 0000:04: resource 7 [io 0x0d00-0x5fff window] :[ 1.608162] pci_bus 0000:04: resource 8 [mem 0x000a0000-0x000bffff window] :[ 1.608164] pci_bus 0000:04: resource 9 [mem 0x000c0000-0x000dffff window] :[ 1.608165] pci_bus 0000:04: resource 10 [mem 0xfed0e000-0xfed0ffff window] :[ 1.608167] pci_bus 0000:04: resource 11 [mem 0x80000000-0xbbffffff window] :[ 1.608179] pci 0000:40:01.0: PCI bridge to [bus 41] :[ 1.608189] pci_bus 0000:40: resource 4 [io 0x6000-0x7fff window] :[ 1.608190] pci_bus 0000:40: resource 5 [mem 0xbc000000-0xcbffffff window] :[ 1.608198] pci_bus 0000:80: resource 4 [io 0x8000-0x9fff window] :[ 1.608200] pci_bus 0000:80: resource 5 [mem 0xcc000000-0xd3ffffff window] :[ 1.608225] pci 0000:c0:02.0: PCI bridge to [bus c1-c2] :[ 1.608228] pci 0000:c0:02.0: bridge window [io 0xf000-0xffff] :[ 1.608232] pci 0000:c0:02.0: bridge window [mem 0xfbe00000-0xfbefffff] :[ 1.608239] pci 0000:c0:02.2: PCI bridge to [bus c3] :[ 1.608249] pci 0000:c0:03.0: PCI bridge to [bus c4] :[ 1.608260] pci_bus 0000:c0: resource 4 [io 0xa000-0xffff window] :[ 1.608262] pci_bus 0000:c0: resource 5 [mem 0xd4000000-0xfbffffff window] :[ 1.608263] pci_bus 0000:c1: resource 0 [io 0xf000-0xffff] :[ 1.608265] pci_bus 0000:c1: resource 1 [mem 0xfbe00000-0xfbefffff] :[ 1.608363] NET: Registered protocol family 2 :[ 1.609225] TCP established hash table entries: 524288 (order: 10, 4194304 bytes) :[ 1.610163] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) :[ 1.610384] TCP: Hash tables configured (established 524288 bind 65536) :[ 1.610435] TCP: reno registered :[ 1.610566] UDP hash table entries: 65536 (order: 9, 2097152 bytes) :[ 1.611115] UDP-Lite hash table entries: 65536 (order: 9, 2097152 bytes) :[ 1.611759] NET: Registered protocol family 1 :[ 1.643835] pci 0000:04:01.0: Boot video device :[ 1.644090] PCI: CLS 64 bytes, default 64 :[ 1.644169] Unpacking initramfs... :[ 1.868413] Freeing initrd memory: 19124k freed :[ 1.879289] PCI-DMA: Using software bounce buffering for IO (SWIOTLB) :[ 1.879299] software IO TLB [mem 0x79f8c000-0x7df8c000] (64MB) mapped at [ffff938979f8c000-ffff93897df8bfff] :[ 1.879420] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 163840 ms ovfl timer :[ 1.879422] RAPL PMU: hw unit of domain pp0-core 2^-16 Joules :[ 1.879423] RAPL PMU: hw unit of domain package 2^-16 Joules :[ 1.879423] RAPL PMU: hw unit of domain dram 2^-16 Joules :[ 1.887546] sha1_ssse3: Using AVX optimized SHA-1 implementation :[ 1.887715] sha256_ssse3: Using AVX optimized SHA-256 implementation :[ 1.888673] futex hash table entries: 8192 (order: 7, 524288 bytes) :[ 1.888823] Initialise system trusted keyring :[ 1.888874] audit: initializing netlink socket (disabled) :[ 1.888901] type=2000 audit(1535484939.700:1): initialized :[ 1.909029] HugeTLB registered 1 GB page size, pre-allocated 0 pages :[ 1.909030] HugeTLB registered 2 MB page size, pre-allocated 0 pages :[ 1.910636] zpool: loaded :[ 1.910638] zbud: loaded :[ 1.910953] VFS: Disk quotas dquot_6.5.2 :[ 1.911006] Dquot-cache hash table entries: 512 (order 0, 4096 bytes) :[ 1.911343] msgmni has been set to 32768 :[ 1.911422] Key type big_key registered :[ 1.911425] SELinux: Registering netfilter hooks :[ 1.913758] NET: Registered protocol family 38 :[ 1.913796] Key type asymmetric registered :[ 1.913804] Asymmetric key parser 'x509' registered :[ 1.913873] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) :[ 1.914065] io scheduler noop registered :[ 1.914069] io scheduler deadline registered (default) :[ 1.914103] io scheduler cfq registered :[ 1.914107] io scheduler mq-deadline registered :[ 1.914110] io scheduler kyber registered :[ 1.914692] pcieport 0000:00:02.0: irq 25 for MSI/MSI-X :[ 1.914905] pcieport 0000:00:03.0: irq 27 for MSI/MSI-X :[ 1.915145] pcieport 0000:00:11.0: irq 28 for MSI/MSI-X :[ 1.915471] pcieport 0000:40:01.0: irq 30 for MSI/MSI-X :[ 1.915800] pcieport 0000:c0:02.0: irq 32 for MSI/MSI-X :[ 1.916033] pcieport 0000:c0:02.2: irq 33 for MSI/MSI-X :[ 1.916262] pcieport 0000:c0:03.0: irq 35 for MSI/MSI-X :[ 1.916440] pcieport 0000:00:02.0: Signaling PME through PCIe PME interrupt :[ 1.916444] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt :[ 1.916447] pcie_pme 0000:00:02.0:pcie001: service driver pcie_pme loaded :[ 1.916464] pcieport 0000:00:03.0: Signaling PME through PCIe PME interrupt :[ 1.916467] pcie_pme 0000:00:03.0:pcie001: service driver pcie_pme loaded :[ 1.916507] pcieport 0000:00:11.0: Signaling PME through PCIe PME interrupt :[ 1.916511] pcie_pme 0000:00:11.0:pcie001: service driver pcie_pme loaded :[ 1.916545] pcieport 0000:40:01.0: Signaling PME through PCIe PME interrupt :[ 1.916548] pcie_pme 0000:40:01.0:pcie001: service driver pcie_pme loaded :[ 1.916571] pcieport 0000:c0:02.0: Signaling PME through PCIe PME interrupt :[ 1.916573] pci 0000:c1:00.0: Signaling PME through PCIe PME interrupt :[ 1.916575] pci 0000:c1:00.1: Signaling PME through PCIe PME interrupt :[ 1.916578] pcie_pme 0000:c0:02.0:pcie001: service driver pcie_pme loaded :[ 1.916597] pcieport 0000:c0:02.2: Signaling PME through PCIe PME interrupt :[ 1.916601] pcie_pme 0000:c0:02.2:pcie001: service driver pcie_pme loaded :[ 1.916627] pcieport 0000:c0:03.0: Signaling PME through PCIe PME interrupt :[ 1.916631] pcie_pme 0000:c0:03.0:pcie001: service driver pcie_pme loaded :[ 1.916831] pci_hotplug: PCI Hot Plug PCI Core version: 0.5 :[ 1.916852] pciehp: PCI Express Hot Plug Controller Driver version: 0.4 :[ 1.916938] intel_idle: MWAIT substates: 0x1120 :[ 1.916939] intel_idle: v0.4.1 model 0x3E :[ 1.917199] intel_idle: lapic_timer_reliable_states 0xffffffff :[ 1.917315] input: Power Button as /devices/LNXSYSTM:00/device:00/PNP0C0C:00/input/input0 :[ 1.917319] ACPI: Power Button [PWRB] :[ 1.917365] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1 :[ 1.917367] ACPI: Power Button [PWRF] :[ 1.917408] ACPI: Requesting acpi_cpufreq :[ 1.948648] ERST: Error Record Serialization Table (ERST) support is initialized. :[ 1.948651] pstore: Registered erst as persistent store backend :[ 1.948709] GHES: APEI firmware first mode is enabled by APEI bit and WHEA _OSC. :[ 1.948845] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled :[ 1.969510] 00:03: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A :[ 1.990218] 00:04: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A :[ 1.990828] Non-volatile memory driver v1.3 :[ 1.990858] Linux agpgart interface v0.103 :[ 2.020781] tpm_tis 00:0f: 1.2 TPM (device-id 0xB, rev-id 16) :[ 2.139932] crash memory driver: version 1.1 :[ 2.140191] rdac: device handler registered :[ 2.140230] hp_sw: device handler registered :[ 2.140232] emc: device handler registered :[ 2.140315] alua: device handler registered :[ 2.140350] libphy: Fixed MDIO Bus: probed :[ 2.140403] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver :[ 2.140407] ehci-pci: EHCI PCI platform driver :[ 2.140606] ehci-pci 0000:00:1a.0: EHCI Host Controller :[ 2.140669] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 1 :[ 2.140681] ehci-pci 0000:00:1a.0: debug port 2 :[ 2.144583] ehci-pci 0000:00:1a.0: cache line size of 64 is not supported :[ 2.144606] ehci-pci 0000:00:1a.0: irq 16, io mem 0xbbb24000 :[ 2.149749] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00 :[ 2.149806] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002 :[ 2.149808] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 :[ 2.149810] usb usb1: Product: EHCI Host Controller :[ 2.149811] usb usb1: Manufacturer: Linux 3.10.0-862.11.6.el7.x86_64 ehci_hcd :[ 2.149813] usb usb1: SerialNumber: 0000:00:1a.0 :[ 2.149921] hub 1-0:1.0: USB hub found :[ 2.149927] hub 1-0:1.0: 2 ports detected :[ 2.150142] ehci-pci 0000:00:1d.0: EHCI Host Controller :[ 2.150187] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 2 :[ 2.150198] ehci-pci 0000:00:1d.0: debug port 2 :[ 2.154104] ehci-pci 0000:00:1d.0: cache line size of 64 is not supported :[ 2.154119] ehci-pci 0000:00:1d.0: irq 23, io mem 0xbbb23000 :[ 2.159751] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00 :[ 2.159794] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002 :[ 2.159796] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 :[ 2.159797] usb usb2: Product: EHCI Host Controller :[ 2.159799] usb usb2: Manufacturer: Linux 3.10.0-862.11.6.el7.x86_64 ehci_hcd :[ 2.159800] usb usb2: SerialNumber: 0000:00:1d.0 :[ 2.159892] hub 2-0:1.0: USB hub found :[ 2.159896] hub 2-0:1.0: 2 ports detected :[ 2.160018] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver :[ 2.160023] ohci-pci: OHCI PCI platform driver :[ 2.160045] uhci_hcd: USB Universal Host Controller Interface driver :[ 2.160118] usbcore: registered new interface driver usbserial :[ 2.160128] usbcore: registered new interface driver usbserial_generic :[ 2.160134] usbserial: USB Serial support registered for generic :[ 2.160159] i8042: PNP: No PS/2 controller found. Probing ports directly. :[ 2.451752] usb 1-1: new high-speed USB device number 2 using ehci-pci :[ 2.461753] usb 2-1: new high-speed USB device number 2 using ehci-pci :[ 2.568149] usb 1-1: New USB device found, idVendor=8087, idProduct=0024 :[ 2.568151] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 :[ 2.568296] hub 1-1:1.0: USB hub found :[ 2.568402] hub 1-1:1.0: 6 ports detected :[ 2.577150] usb 2-1: New USB device found, idVendor=8087, idProduct=0024 :[ 2.577152] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 :[ 2.577294] hub 2-1:1.0: USB hub found :[ 2.577399] hub 2-1:1.0: 8 ports detected :[ 2.852758] usb 2-1.4: new full-speed USB device number 3 using ehci-pci :[ 2.882824] tsc: Refined TSC clocksource calibration: 3299.999 MHz :[ 3.199141] i8042: No controller found :[ 3.199241] Switched to clocksource tsc :[ 3.199289] mousedev: PS/2 mouse device common for all mice :[ 3.199572] rtc_cmos 00:05: RTC can wake from S4 :[ 3.199708] rtc_cmos 00:05: rtc core: registered rtc_cmos as rtc0 :[ 3.199742] rtc_cmos 00:05: alarms up to one month, y3k, 114 bytes nvram, hpet irqs :[ 3.199797] intel_pstate: Intel P-state driver initializing :[ 3.204415] cpuidle: using governor menu :[ 3.206372] hidraw: raw HID events driver (C) Jiri Kosina :[ 3.206681] usbcore: registered new interface driver usbhid :[ 3.206684] usbhid: USB HID core driver :[ 3.207021] drop_monitor: Initializing network drop monitor service :[ 3.207276] TCP: cubic registered :[ 3.207287] Initializing XFRM netlink socket :[ 3.207696] NET: Registered protocol family 10 :[ 3.208434] NET: Registered protocol family 17 :[ 3.208452] mpls_gso: MPLS GSO support :[ 3.210869] microcode: sig=0x306e4, pf=0x20, revision=0x42d :[ 3.211364] microcode: Microcode Update Driver: v2.01 , Peter Oruba :[ 3.211656] PM: Hibernation image not present or could not be loaded. :[ 3.211665] Loading compiled-in X.509 certificates :[ 3.212296] usb 2-1.4: New USB device found, idVendor=0557, idProduct=2221 :[ 3.212303] usb 2-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=0 :[ 3.212307] usb 2-1.4: Product: Hermon USB hidmouse Device :[ 3.212311] usb 2-1.4: Manufacturer: Winbond Electronics Corp :[ 3.212748] Loaded X.509 cert 'Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87362bc7229d9f465321773dfd1f77a80' :[ 3.213751] input: Winbond Electronics Corp Hermon USB hidmouse Device as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4:1.0/input/input2 :[ 3.213802] Loaded X.509 cert 'Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b72e3852e2014c3a676fc8' :[ 3.214045] hid-generic 0003:0557:2221.0001: input,hidraw0: USB HID v1.00 Mouse [Winbond Electronics Corp Hermon USB hidmouse Device] on usb-0000:00:1d.0-1.4/input0 :[ 3.214751] Loaded X.509 cert 'Red Hat Enterprise Linux kernel signing key: 4e9b1987b4e91df16c0234ff72721b35c22ea979' :[ 3.214807] registered taskstats version 1 :[ 3.214915] input: Winbond Electronics Corp Hermon USB hidmouse Device as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.4/2-1.4:1.1/input/input3 :[ 3.220179] Key type trusted registered :[ 3.224412] Key type encrypted registered :[ 3.266206] hid-generic 0003:0557:2221.0002: input,hidraw1: USB HID v1.00 Keyboard [Winbond Electronics Corp Hermon USB hidmouse Device] on usb-0000:00:1d.0-1.4/input1 :[ 3.484092] Magic number: 14:163:598 :[ 3.484253] pci 0000:7f:16.1: hash matches :[ 3.484282] pci 0000:3f:09.0: hash matches :[ 3.484934] memory memory1820: hash matches :[ 3.485046] memory memory1374: hash matches :[ 3.485227] memory memory587: hash matches :[ 3.485454] rtc_cmos 00:05: setting system clock to 2018-08-28 19:35:42 UTC (1535484942) :[ 3.487119] Freeing unused kernel memory: 1836k freed :[ 3.488043] Write protecting the kernel read-only data: 12288k :[ 3.491601] Freeing unused kernel memory: 832k freed :[ 3.494666] Freeing unused kernel memory: 676k freed :[ 3.505420] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN) :[ 3.507290] systemd[1]: Detected architecture x86-64. :[ 3.507297] systemd[1]: Running in initial RAM disk. :[ 3.507365] systemd[1]: Set hostname to . :[ 3.576947] systemd[1]: Reached target Local File Systems. :[ 3.576966] systemd[1]: Starting Local File Systems. :[ 3.576985] systemd[1]: Reached target Timers. :[ 3.576996] systemd[1]: Starting Timers. :[ 3.577268] systemd[1]: Created slice Root Slice. :[ 3.577283] systemd[1]: Starting Root Slice. :[ 3.577425] systemd[1]: Created slice System Slice. :[ 3.577441] systemd[1]: Starting System Slice. :[ 3.577507] systemd[1]: Listening on udev Kernel Socket. :[ 3.577520] systemd[1]: Starting udev Kernel Socket. :[ 3.577582] systemd[1]: Listening on udev Control Socket. :[ 3.577595] systemd[1]: Starting udev Control Socket. :[ 3.577617] systemd[1]: Reached target Slices. :[ 3.577628] systemd[1]: Starting Slices. :[ 3.577724] systemd[1]: Listening on Journal Socket. :[ 3.577740] systemd[1]: Starting Journal Socket. :[ 3.577755] systemd[1]: Reached target Sockets. :[ 3.577788] systemd[1]: Starting Sockets. :[ 3.578652] systemd[1]: Starting Journal Service... :[ 3.579422] systemd[1]: Starting Setup Virtual Console... :[ 3.580492] systemd[1]: Starting Create list of required static device nodes for the current kernel... :[ 3.581598] systemd[1]: Starting Apply Kernel Variables... :[ 3.582690] systemd[1]: Starting dracut cmdline hook... :[ 3.582752] systemd[1]: Reached target Swap. :[ 3.582791] systemd[1]: Starting Swap. :[ 3.583525] systemd[1]: Started Create list of required static device nodes for the current kernel. :[ 3.584860] systemd[1]: Starting Create Static Device Nodes in /dev... :[ 3.585534] systemd[1]: Started Apply Kernel Variables. :[ 3.589439] systemd[1]: Started Create Static Device Nodes in /dev. :[ 3.656842] systemd[1]: Started Setup Virtual Console. :[ 3.663540] systemd[1]: Started Journal Service. :[ 4.059527] dca service started, version 1.12.1 :[ 4.061713] pps_core: LinuxPPS API ver. 1 registered :[ 4.061717] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti :[ 4.062833] libata version 3.00 loaded. :[ 4.062878] mpt3sas version 16.100.00.00 loaded :[ 4.063254] mpt3sas_cm0: 64 BIT PCI BUS DMA ADDRESSING SUPPORTED, total mem (528164672 kB) :[ 4.063976] PTP clock support registered :[ 4.083011] ahci 0000:00:1f.2: version 3.0 :[ 4.083231] ahci 0000:00:1f.2: irq 36 for MSI/MSI-X :[ 4.084569] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.4.0-k :[ 4.084572] igb: Copyright (c) 2007-2014 Intel Corporation. :[ 4.093811] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3f impl SATA mode :[ 4.093817] ahci 0000:00:1f.2: flags: 64bit ncq sntf pm led clo pio slum part ems apst :[ 4.103043] igb 0000:c1:00.0: irq 37 for MSI/MSI-X :[ 4.103094] igb 0000:c1:00.0: irq 37 for MSI/MSI-X :[ 4.103108] igb 0000:c1:00.0: irq 38 for MSI/MSI-X :[ 4.103116] igb 0000:c1:00.0: irq 39 for MSI/MSI-X :[ 4.103124] igb 0000:c1:00.0: irq 40 for MSI/MSI-X :[ 4.103132] igb 0000:c1:00.0: irq 41 for MSI/MSI-X :[ 4.103140] igb 0000:c1:00.0: irq 42 for MSI/MSI-X :[ 4.103148] igb 0000:c1:00.0: irq 43 for MSI/MSI-X :[ 4.103158] igb 0000:c1:00.0: irq 44 for MSI/MSI-X :[ 4.103174] igb 0000:c1:00.0: irq 45 for MSI/MSI-X :[ 4.105690] scsi host1: ahci :[ 4.106025] scsi host2: ahci :[ 4.106355] scsi host3: ahci :[ 4.106491] scsi host4: ahci :[ 4.106642] scsi host5: ahci :[ 4.106796] scsi host6: ahci :[ 4.106972] ata1: SATA max UDMA/133 abar m2048 at 0xbbb22000 port 0xbbb22100 irq 36 :[ 4.106975] ata2: SATA max UDMA/133 abar m2048 at 0xbbb22000 port 0xbbb22180 irq 36 :[ 4.106977] ata3: SATA max UDMA/133 abar m2048 at 0xbbb22000 port 0xbbb22200 irq 36 :[ 4.106980] ata4: SATA max UDMA/133 abar m2048 at 0xbbb22000 port 0xbbb22280 irq 36 :[ 4.106982] ata5: SATA max UDMA/133 abar m2048 at 0xbbb22000 port 0xbbb22300 irq 36 :[ 4.106987] ata6: SATA max UDMA/133 abar m2048 at 0xbbb22000 port 0xbbb22380 irq 36 :[ 4.119161] [TTM] Zone kernel: Available graphics memory: 264082336 kiB :[ 4.119163] [TTM] Zone dma32: Available graphics memory: 2097152 kiB :[ 4.119165] [TTM] Initializing pool allocator :[ 4.119171] [TTM] Initializing DMA pool allocator :[ 4.128640] mpt3sas_cm0: MSI-X vectors supported: 96, no of cores: 32, max_msix_vectors: -1 :[ 4.128682] mpt3sas 0000:01:00.0: irq 46 for MSI/MSI-X :[ 4.128691] mpt3sas 0000:01:00.0: irq 47 for MSI/MSI-X :[ 4.128706] mpt3sas 0000:01:00.0: irq 48 for MSI/MSI-X :[ 4.128714] mpt3sas 0000:01:00.0: irq 49 for MSI/MSI-X :[ 4.128723] mpt3sas 0000:01:00.0: irq 50 for MSI/MSI-X :[ 4.128731] mpt3sas 0000:01:00.0: irq 51 for MSI/MSI-X :[ 4.128739] mpt3sas 0000:01:00.0: irq 52 for MSI/MSI-X :[ 4.128747] mpt3sas 0000:01:00.0: irq 53 for MSI/MSI-X :[ 4.128756] mpt3sas 0000:01:00.0: irq 54 for MSI/MSI-X :[ 4.128764] mpt3sas 0000:01:00.0: irq 55 for MSI/MSI-X :[ 4.128791] mpt3sas 0000:01:00.0: irq 56 for MSI/MSI-X :[ 4.128799] mpt3sas 0000:01:00.0: irq 57 for MSI/MSI-X :[ 4.128809] mpt3sas 0000:01:00.0: irq 58 for MSI/MSI-X :[ 4.128817] mpt3sas 0000:01:00.0: irq 59 for MSI/MSI-X :[ 4.128826] mpt3sas 0000:01:00.0: irq 60 for MSI/MSI-X :[ 4.128835] mpt3sas 0000:01:00.0: irq 61 for MSI/MSI-X :[ 4.128844] mpt3sas 0000:01:00.0: irq 62 for MSI/MSI-X :[ 4.128852] mpt3sas 0000:01:00.0: irq 63 for MSI/MSI-X :[ 4.128867] mpt3sas 0000:01:00.0: irq 64 for MSI/MSI-X :[ 4.128875] mpt3sas 0000:01:00.0: irq 65 for MSI/MSI-X :[ 4.128884] mpt3sas 0000:01:00.0: irq 66 for MSI/MSI-X :[ 4.128892] mpt3sas 0000:01:00.0: irq 67 for MSI/MSI-X :[ 4.128900] mpt3sas 0000:01:00.0: irq 68 for MSI/MSI-X :[ 4.128909] mpt3sas 0000:01:00.0: irq 69 for MSI/MSI-X :[ 4.128917] mpt3sas 0000:01:00.0: irq 70 for MSI/MSI-X :[ 4.128926] mpt3sas 0000:01:00.0: irq 71 for MSI/MSI-X :[ 4.128945] mpt3sas 0000:01:00.0: irq 72 for MSI/MSI-X :[ 4.128953] mpt3sas 0000:01:00.0: irq 73 for MSI/MSI-X :[ 4.128962] mpt3sas 0000:01:00.0: irq 74 for MSI/MSI-X :[ 4.128971] mpt3sas 0000:01:00.0: irq 75 for MSI/MSI-X :[ 4.128979] mpt3sas 0000:01:00.0: irq 76 for MSI/MSI-X :[ 4.128987] mpt3sas 0000:01:00.0: irq 77 for MSI/MSI-X :[ 4.129449] mpt3sas0-msix0: PCI-MSI-X enabled: IRQ 46 :[ 4.129451] mpt3sas0-msix1: PCI-MSI-X enabled: IRQ 47 :[ 4.129452] mpt3sas0-msix2: PCI-MSI-X enabled: IRQ 48 :[ 4.129454] mpt3sas0-msix3: PCI-MSI-X enabled: IRQ 49 :[ 4.129455] mpt3sas0-msix4: PCI-MSI-X enabled: IRQ 50 :[ 4.129456] mpt3sas0-msix5: PCI-MSI-X enabled: IRQ 51 :[ 4.129457] mpt3sas0-msix6: PCI-MSI-X enabled: IRQ 52 :[ 4.129458] mpt3sas0-msix7: PCI-MSI-X enabled: IRQ 53 :[ 4.129459] mpt3sas0-msix8: PCI-MSI-X enabled: IRQ 54 :[ 4.129461] mpt3sas0-msix9: PCI-MSI-X enabled: IRQ 55 :[ 4.129462] mpt3sas0-msix10: PCI-MSI-X enabled: IRQ 56 :[ 4.129463] mpt3sas0-msix11: PCI-MSI-X enabled: IRQ 57 :[ 4.129464] mpt3sas0-msix12: PCI-MSI-X enabled: IRQ 58 :[ 4.129466] mpt3sas0-msix13: PCI-MSI-X enabled: IRQ 59 :[ 4.129467] mpt3sas0-msix14: PCI-MSI-X enabled: IRQ 60 :[ 4.129468] mpt3sas0-msix15: PCI-MSI-X enabled: IRQ 61 :[ 4.129469] mpt3sas0-msix16: PCI-MSI-X enabled: IRQ 62 :[ 4.129470] mpt3sas0-msix17: PCI-MSI-X enabled: IRQ 63 :[ 4.129471] mpt3sas0-msix18: PCI-MSI-X enabled: IRQ 64 :[ 4.129473] mpt3sas0-msix19: PCI-MSI-X enabled: IRQ 65 :[ 4.129474] mpt3sas0-msix20: PCI-MSI-X enabled: IRQ 66 :[ 4.129475] mpt3sas0-msix21: PCI-MSI-X enabled: IRQ 67 :[ 4.129476] mpt3sas0-msix22: PCI-MSI-X enabled: IRQ 68 :[ 4.129477] mpt3sas0-msix23: PCI-MSI-X enabled: IRQ 69 :[ 4.129479] mpt3sas0-msix24: PCI-MSI-X enabled: IRQ 70 :[ 4.129480] mpt3sas0-msix25: PCI-MSI-X enabled: IRQ 71 :[ 4.129481] mpt3sas0-msix26: PCI-MSI-X enabled: IRQ 72 :[ 4.129482] mpt3sas0-msix27: PCI-MSI-X enabled: IRQ 73 :[ 4.129484] mpt3sas0-msix28: PCI-MSI-X enabled: IRQ 74 :[ 4.129485] mpt3sas0-msix29: PCI-MSI-X enabled: IRQ 75 :[ 4.129486] mpt3sas0-msix30: PCI-MSI-X enabled: IRQ 76 :[ 4.129487] mpt3sas0-msix31: PCI-MSI-X enabled: IRQ 77 :[ 4.129489] mpt3sas_cm0: iomem(0x00000000bba40000), mapped(0xffffba7db1c80000), size(65536) :[ 4.129491] mpt3sas_cm0: ioport(0x0000000000004000), size(256) :[ 4.149263] fbcon: mgadrmfb (fb0) is primary device :[ 4.156940] igb 0000:c1:00.0: added PHC on eth0 :[ 4.156941] igb 0000:c1:00.0: Intel(R) Gigabit Ethernet Network Connection :[ 4.156943] igb 0000:c1:00.0: eth0: (PCIe:5.0Gb/s:Width x4) 00:25:90:5c:f1:9c :[ 4.157016] igb 0000:c1:00.0: eth0: PBA No: 104900-000 :[ 4.157017] igb 0000:c1:00.0: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) :[ 4.165647] igb 0000:c1:00.1: irq 79 for MSI/MSI-X :[ 4.165687] igb 0000:c1:00.1: irq 79 for MSI/MSI-X :[ 4.165700] igb 0000:c1:00.1: irq 80 for MSI/MSI-X :[ 4.165707] igb 0000:c1:00.1: irq 81 for MSI/MSI-X :[ 4.165715] igb 0000:c1:00.1: irq 82 for MSI/MSI-X :[ 4.165721] igb 0000:c1:00.1: irq 83 for MSI/MSI-X :[ 4.165728] igb 0000:c1:00.1: irq 84 for MSI/MSI-X :[ 4.165734] igb 0000:c1:00.1: irq 85 for MSI/MSI-X :[ 4.165741] igb 0000:c1:00.1: irq 86 for MSI/MSI-X :[ 4.165747] igb 0000:c1:00.1: irq 87 for MSI/MSI-X :[ 4.221652] igb 0000:c1:00.1: added PHC on eth1 :[ 4.221653] igb 0000:c1:00.1: Intel(R) Gigabit Ethernet Network Connection :[ 4.221655] igb 0000:c1:00.1: eth1: (PCIe:5.0Gb/s:Width x4) 00:25:90:5c:f1:9d :[ 4.221728] igb 0000:c1:00.1: eth1: PBA No: 104900-000 :[ 4.221729] igb 0000:c1:00.1: Using MSI-X interrupts. 8 rx queue(s), 8 tx queue(s) :[ 4.411823] ata5: SATA link down (SStatus 0 SControl 300) :[ 4.411859] ata3: SATA link down (SStatus 0 SControl 300) :[ 4.411882] ata4: SATA link down (SStatus 0 SControl 300) :[ 4.411905] ata1: SATA link down (SStatus 0 SControl 300) :[ 4.411929] ata6: SATA link down (SStatus 0 SControl 300) :[ 4.411952] ata2: SATA link down (SStatus 0 SControl 300) :[ 4.596981] Console: switching to colour frame buffer device 128x48 :[ 4.665573] mgag200 0000:04:01.0: fb0: mgadrmfb frame buffer device :[ 4.669822] mpt3sas_cm0: sending message unit reset !! :[ 4.670920] [drm] Initialized mgag200 1.0.0 20110418 for 0000:04:01.0 on minor 0 :[ 4.671332] mpt3sas_cm0: message unit reset: SUCCESS :[ 4.759614] mpt3sas_cm0: Allocated physical memory: size(21292 kB) :[ 4.759615] mpt3sas_cm0: Current Controller Queue Depth(10104),Max Controller Queue Depth(10240) :[ 4.759616] mpt3sas_cm0: Scatter Gather Elements per IO(128) :[ 4.808217] mpt3sas_cm0: LSISAS3008: FWVersion(08.00.00.00), ChipRevision(0x02), BiosVersion(08.09.02.00) :[ 4.808219] mpt3sas_cm0: Protocol=( :[ 4.808220] Initiator :[ 4.808220] ,Target :[ 4.808221] ), :[ 4.808222] Capabilities=( :[ 4.808222] TLR :[ 4.808223] ,EEDP :[ 4.808224] ,Snapshot Buffer :[ 4.808225] ,Diag Trace Buffer :[ 4.808225] ,Task Set Full :[ 4.808226] ,NCQ :[ 4.808226] ) :[ 4.808335] scsi host0: Fusion MPT SAS Host :[ 4.808840] mpt3sas_cm0: sending port enable !! :[ 4.812124] mpt3sas_cm0: host_add: handle(0x0001), sas_addr(0x500605b00a2392e0), phys(8) :[ 4.819806] mpt3sas_cm0: port enable: SUCCESS :[ 4.822052] scsi 0:0:0:0: Direct-Access ATA ST1000NM0033-9ZM SN04 PQ: 0 ANSI: 6 :[ 4.822064] scsi 0:0:0:0: SATA: handle(0x000a), sas_addr(0x4433221102000000), phy(2), device_name(0x5000c5007aa837bc) :[ 4.822069] scsi 0:0:0:0: enclosure logical id (0x500605b00a2392e0), slot(0) :[ 4.822072] scsi 0:0:0:0: enclosure level(0x0000), connector name( ) :[ 4.822122] scsi 0:0:0:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y) :[ 4.822132] scsi 0:0:0:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) :[ 4.837364] scsi 0:0:1:0: Direct-Access ATA ST4000NM0033-9ZM SN04 PQ: 0 ANSI: 6 :[ 4.837374] scsi 0:0:1:0: SATA: handle(0x0009), sas_addr(0x4433221101000000), phy(1), device_name(0x5000c5007a7a4be7) :[ 4.837378] scsi 0:0:1:0: enclosure logical id (0x500605b00a2392e0), slot(2) :[ 4.837382] scsi 0:0:1:0: enclosure level(0x0000), connector name( ) :[ 4.837430] scsi 0:0:1:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y) :[ 4.837435] scsi 0:0:1:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) :[ 4.852194] scsi 0:0:2:0: Direct-Access ATA ST1000NM0033-9ZM SN04 PQ: 0 ANSI: 6 :[ 4.852205] scsi 0:0:2:0: SATA: handle(0x000b), sas_addr(0x4433221103000000), phy(3), device_name(0x5000c5007aac175f) :[ 4.852209] scsi 0:0:2:0: enclosure logical id (0x500605b00a2392e0), slot(1) :[ 4.852212] scsi 0:0:2:0: enclosure level(0x0000), connector name( ) :[ 4.852260] scsi 0:0:2:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y) :[ 4.852268] scsi 0:0:2:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) :[ 4.870326] scsi 0:0:3:0: Direct-Access ATA ST4000NM0033-9ZM SN04 PQ: 0 ANSI: 6 :[ 4.870337] scsi 0:0:3:0: SATA: handle(0x000c), sas_addr(0x4433221105000000), phy(5), device_name(0x5000c5007a79df13) :[ 4.870343] scsi 0:0:3:0: enclosure logical id (0x500605b00a2392e0), slot(6) :[ 4.870347] scsi 0:0:3:0: enclosure level(0x0000), connector name( ) :[ 4.870394] scsi 0:0:3:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y) :[ 4.870400] scsi 0:0:3:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) :[ 4.888336] scsi 0:0:4:0: Direct-Access ATA ST4000NM0033-9ZM SN04 PQ: 0 ANSI: 6 :[ 4.888347] scsi 0:0:4:0: SATA: handle(0x000d), sas_addr(0x4433221106000000), phy(6), device_name(0x5000c5007a7a64b7) :[ 4.888351] scsi 0:0:4:0: enclosure logical id (0x500605b00a2392e0), slot(4) :[ 4.888355] scsi 0:0:4:0: enclosure level(0x0000), connector name( ) :[ 4.888404] scsi 0:0:4:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y) :[ 4.888411] scsi 0:0:4:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) :[ 4.909416] scsi 0:0:5:0: Direct-Access ATA ST4000NM0033-9ZM SN04 PQ: 0 ANSI: 6 :[ 4.909430] scsi 0:0:5:0: SATA: handle(0x000e), sas_addr(0x4433221107000000), phy(7), device_name(0x5000c5007a7a71a9) :[ 4.909436] scsi 0:0:5:0: enclosure logical id (0x500605b00a2392e0), slot(5) :[ 4.909440] scsi 0:0:5:0: enclosure level(0x0000), connector name( ) :[ 4.909492] scsi 0:0:5:0: atapi(n), ncq(y), asyn_notify(n), smart(y), fua(y), sw_preserve(y) :[ 4.909500] scsi 0:0:5:0: qdepth(32), tagged(1), simple(0), ordered(0), scsi_level(7), cmd_que(1) :[ 4.947648] sd 0:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) :[ 4.947757] sd 0:0:1:0: [sdb] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB) :[ 4.947775] sd 0:0:2:0: [sdc] 1953525168 512-byte logical blocks: (1.00 TB/931 GiB) :[ 4.947910] sd 0:0:3:0: [sdd] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB) :[ 4.947933] sd 0:0:4:0: [sde] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB) :[ 4.948067] sd 0:0:5:0: [sdf] 7814037168 512-byte logical blocks: (4.00 TB/3.63 TiB) :[ 5.484311] sd 0:0:5:0: [sdf] Write Protect is off :[ 5.484321] sd 0:0:5:0: [sdf] Mode Sense: 9b 00 10 08 :[ 5.485834] sd 0:0:5:0: [sdf] Write cache: enabled, read cache: enabled, supports DPO and FUA :[ 5.489608] sd 0:0:2:0: [sdc] Write Protect is off :[ 5.489615] sd 0:0:2:0: [sdc] Mode Sense: 9b 00 10 08 :[ 5.490650] sd 0:0:2:0: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA :[ 5.494307] sd 0:0:3:0: [sdd] Write Protect is off :[ 5.494313] sd 0:0:3:0: [sdd] Mode Sense: 9b 00 10 08 :[ 5.495712] sd 0:0:3:0: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA :[ 5.498454] sd 0:0:4:0: [sde] Write Protect is off :[ 5.498459] sd 0:0:4:0: [sde] Mode Sense: 9b 00 10 08 :[ 5.499158] sd 0:0:1:0: [sdb] Write Protect is off :[ 5.499163] sd 0:0:1:0: [sdb] Mode Sense: 9b 00 10 08 :[ 5.499490] sd 0:0:0:0: [sda] Write Protect is off :[ 5.499498] sd 0:0:0:0: [sda] Mode Sense: 9b 00 10 08 :[ 5.499702] sd 0:0:4:0: [sde] Write cache: enabled, read cache: enabled, supports DPO and FUA :[ 5.500166] sd 0:0:1:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA :[ 5.501300] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA :[ 5.995473] sdc: sdc1 sdc2 sdc3 :[ 6.025997] sda: sda1 sda2 sda3 sda4 < sda5 > :[ 6.544539] sd 0:0:5:0: [sdf] Attached SCSI disk :[ 6.549629] sd 0:0:2:0: [sdc] Attached SCSI disk :[ 6.554438] sd 0:0:3:0: [sdd] Attached SCSI disk :[ 6.558554] sd 0:0:4:0: [sde] Attached SCSI disk :[ 6.559245] sd 0:0:1:0: [sdb] Attached SCSI disk :[ 6.576465] sd 0:0:0:0: [sda] Attached SCSI disk :[ 6.628163] random: crng init done :[ 6.828184] md/raid1:md126: active with 2 out of 2 mirrors :[ 6.831727] md/raid1:md127: active with 2 out of 2 mirrors :[ 6.846311] md126: detected capacity change from 0 to 1000275968 :[ 6.852484] md127: detected capacity change from 0 to 59999518720 :[ 7.107961] SGI XFS with ACLs, security attributes, no debug enabled :[ 7.112085] XFS (md127): Mounting V4 Filesystem :[ 7.225780] XFS (md127): Ending clean mount :[ 7.630407] systemd-journald[286]: Received SIGTERM from PID 1 (systemd). :[ 8.153813] type=1404 audit(1535484947.166:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295 :[ 8.239656] SELinux: 2048 avtab hash slots, 106487 rules. :[ 8.317706] SELinux: 2048 avtab hash slots, 106487 rules. :[ 8.351830] SELinux: 8 users, 14 roles, 5009 types, 310 bools, 1 sens, 1024 cats :[ 8.351834] SELinux: 97 classes, 106487 rules :[ 8.356978] SELinux: Completing initialization. :[ 8.356980] SELinux: Setting up existing superblocks. :[ 8.371208] type=1403 audit(1535484947.384:3): policy loaded auid=4294967295 ses=4294967295 :[ 8.377441] systemd[1]: Successfully loaded SELinux policy in 229.033ms. :[ 8.761324] ip_tables: (C) 2000-2006 Netfilter Core Team :[ 8.761476] systemd[1]: Inserted module 'ip_tables' :[ 8.809137] systemd[1]: Relabelled /dev and /run in 44.717ms. :[ 10.616830] systemd-journald[748]: Received request to flush runtime journal from PID 1 :[ 12.016352] wmi: Mapper loaded :[ 12.034055] mei_me 0000:00:16.0: Device doesn't have valid ME Interface :[ 12.042072] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 :[ 12.042392] ioatdma: Intel(R) QuickData Technology Driver 4.00 :[ 12.042808] ioatdma 0000:00:04.0: irq 89 for MSI/MSI-X :[ 12.043544] ioatdma 0000:00:04.1: irq 91 for MSI/MSI-X :[ 12.043958] ioatdma 0000:00:04.2: irq 92 for MSI/MSI-X :[ 12.044322] ioatdma 0000:00:04.3: irq 93 for MSI/MSI-X :[ 12.044831] ioatdma 0000:00:04.4: irq 94 for MSI/MSI-X :[ 12.045525] ioatdma 0000:00:04.5: irq 95 for MSI/MSI-X :[ 12.046279] ioatdma 0000:00:04.6: irq 96 for MSI/MSI-X :[ 12.046871] ioatdma 0000:00:04.7: irq 97 for MSI/MSI-X :[ 12.047611] ioatdma 0000:40:04.0: irq 99 for MSI/MSI-X :[ 12.048196] ioatdma 0000:40:04.1: irq 101 for MSI/MSI-X :[ 12.048534] ioatdma 0000:40:04.2: irq 102 for MSI/MSI-X :[ 12.048801] ioatdma 0000:40:04.3: irq 103 for MSI/MSI-X :[ 12.049282] ioatdma 0000:40:04.4: irq 104 for MSI/MSI-X :[ 12.049794] ioatdma 0000:40:04.5: irq 105 for MSI/MSI-X :[ 12.050432] ioatdma 0000:40:04.6: irq 106 for MSI/MSI-X :[ 12.051425] ioatdma 0000:40:04.7: irq 107 for MSI/MSI-X :[ 12.052077] ioatdma 0000:80:04.0: irq 109 for MSI/MSI-X :[ 12.052511] ioatdma 0000:80:04.1: irq 111 for MSI/MSI-X :[ 12.052814] ioatdma 0000:80:04.2: irq 112 for MSI/MSI-X :[ 12.053213] ioatdma 0000:80:04.3: irq 113 for MSI/MSI-X :[ 12.053607] ioatdma 0000:80:04.4: irq 114 for MSI/MSI-X :[ 12.053938] ioatdma 0000:80:04.5: irq 115 for MSI/MSI-X :[ 12.054252] ioatdma 0000:80:04.6: irq 116 for MSI/MSI-X :[ 12.054484] ioatdma 0000:80:04.7: irq 117 for MSI/MSI-X :[ 12.054971] ioatdma 0000:c0:04.0: irq 119 for MSI/MSI-X :[ 12.055600] igb 0000:c1:00.0: DCA enabled :[ 12.055736] igb 0000:c1:00.1: DCA enabled :[ 12.055912] ioatdma 0000:c0:04.1: irq 125 for MSI/MSI-X :[ 12.056425] ioatdma 0000:c0:04.2: irq 126 for MSI/MSI-X :[ 12.056814] ioatdma 0000:c0:04.3: irq 127 for MSI/MSI-X :[ 12.057199] ioatdma 0000:c0:04.4: irq 128 for MSI/MSI-X :[ 12.057692] ioatdma 0000:c0:04.5: irq 129 for MSI/MSI-X :[ 12.058032] ioatdma 0000:c0:04.6: irq 130 for MSI/MSI-X :[ 12.058365] ioatdma 0000:c0:04.7: irq 131 for MSI/MSI-X :[ 12.084797] input: PC Speaker as /devices/platform/pcspkr/input/input4 :[ 12.119522] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt :[ 12.125500] sd 0:0:0:0: Attached scsi generic sg0 type 0 :[ 12.125720] sd 0:0:1:0: Attached scsi generic sg1 type 0 :[ 12.126035] sd 0:0:2:0: Attached scsi generic sg2 type 0 :[ 12.126285] sd 0:0:3:0: Attached scsi generic sg3 type 0 :[ 12.126789] sd 0:0:4:0: Attached scsi generic sg4 type 0 :[ 12.127057] sd 0:0:5:0: Attached scsi generic sg5 type 0 :[ 12.141542] ipmi message handler version 39.2 :[ 12.143865] ipmi device interface :[ 12.158507] ipmi_si IPI0001:00: ipmi_si: probing via ACPI :[ 12.158523] ipmi_si IPI0001:00: [io 0x0ca2] regsize 1 spacing 1 irq 0 :[ 12.158524] ipmi_si: Adding ACPI-specified kcs state machine :[ 12.158565] IPMI System Interface driver. :[ 12.158640] ipmi_si: probing via SMBIOS :[ 12.158641] ipmi_si: SMBIOS: io 0xca2 regsize 1 spacing 1 irq 0 :[ 12.158643] (NULL device *): SMBIOS-specified kcs state machine: duplicate :[ 12.158644] ipmi_si: probing via SPMI :[ 12.158645] ipmi_si: SPMI: io 0xca2 regsize 1 spacing 1 irq 0 :[ 12.158646] (NULL device *): SPMI-specified kcs state machine: duplicate :[ 12.158648] ipmi_si: Trying ACPI-specified kcs state machine at i/o address 0xca2, slave address 0x20, irq 0 :[ 12.184899] ipmi_si IPI0001:00: The BMC does not support clearing the recv irq bit, compensating, but the BMC needs to be fixed. :[ 12.196245] ipmi_si IPI0001:00: Found new BMC (man_id: 0x002a7c, prod_id: 0x0707, dev_id: 0x20) :[ 12.196428] ipmi_si IPI0001:00: IPMI kcs interface initialized :[ 12.330210] cryptd: max_cpu_qlen set to 100 :[ 12.450844] async_tx: api initialized (async) :[ 12.456865] AVX version of gcm_enc/dec engaged. :[ 12.456868] AES CTR mode by8 optimization enabled :[ 12.460780] alg: No test for __gcm-aes-aesni (__driver-gcm-aes-aesni) :[ 12.460825] alg: No test for __generic-gcm-aes-aesni (__driver-generic-gcm-aes-aesni) :[ 12.511321] xor: automatically using best checksumming function: :[ 12.520887] avx : 23688.000 MB/sec :[ 12.572889] raid6: sse2x1 gen() 8050 MB/s :[ 12.589891] raid6: sse2x2 gen() 10242 MB/s :[ 12.606893] raid6: sse2x4 gen() 12375 MB/s :[ 12.606895] raid6: using algorithm sse2x4 gen() (12375 MB/s) :[ 12.606896] raid6: using ssse3x2 recovery algorithm :[ 12.639714] iTCO_vendor_support: vendor-support=0 :[ 12.674245] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11 :[ 12.674319] iTCO_wdt: unable to reset NO_REBOOT flag, device disabled by hardware/BIOS :[ 12.790141] intel_rapl: Found RAPL domain package :[ 12.790147] intel_rapl: Found RAPL domain core :[ 12.790153] intel_rapl: Found RAPL domain dram :[ 12.790184] intel_rapl: Found RAPL domain package :[ 12.790186] intel_rapl: Found RAPL domain core :[ 12.790193] intel_rapl: Found RAPL domain dram :[ 12.790217] intel_rapl: Found RAPL domain package :[ 12.790220] intel_rapl: Found RAPL domain core :[ 12.790228] intel_rapl: Found RAPL domain dram :[ 12.790264] intel_rapl: Found RAPL domain package :[ 12.790269] intel_rapl: Found RAPL domain core :[ 12.790280] intel_rapl: Found RAPL domain dram :[ 12.796560] md/raid:md0: device sdb operational as raid disk 0 :[ 12.796565] md/raid:md0: device sdd operational as raid disk 3 :[ 12.796566] md/raid:md0: device sdf operational as raid disk 2 :[ 12.796568] md/raid:md0: device sde operational as raid disk 1 :[ 12.797216] md/raid:md0: raid level 6 active with 4 out of 4 devices, algorithm 2 :[ 12.806978] md0: detected capacity change from 0 to 8001304920064 :[ 12.931008] EDAC sbridge: Seeking for: PCI ID 8086:0ea0 :[ 12.931034] EDAC sbridge: Seeking for: PCI ID 8086:0ea0 :[ 12.931049] EDAC sbridge: Seeking for: PCI ID 8086:0ea0 :[ 12.931062] EDAC sbridge: Seeking for: PCI ID 8086:0ea0 :[ 12.931076] EDAC sbridge: Seeking for: PCI ID 8086:0ea0 :[ 12.931080] EDAC sbridge: Seeking for: PCI ID 8086:0ea8 :[ 12.931087] EDAC sbridge: Seeking for: PCI ID 8086:0ea8 :[ 12.931094] EDAC sbridge: Seeking for: PCI ID 8086:0ea8 :[ 12.931102] EDAC sbridge: Seeking for: PCI ID 8086:0ea8 :[ 12.931110] EDAC sbridge: Seeking for: PCI ID 8086:0ea8 :[ 12.931113] EDAC sbridge: Seeking for: PCI ID 8086:0e71 :[ 12.931120] EDAC sbridge: Seeking for: PCI ID 8086:0e71 :[ 12.931127] EDAC sbridge: Seeking for: PCI ID 8086:0e71 :[ 12.931148] EDAC sbridge: Seeking for: PCI ID 8086:0e71 :[ 12.931156] EDAC sbridge: Seeking for: PCI ID 8086:0e71 :[ 12.931159] EDAC sbridge: Seeking for: PCI ID 8086:0eaa :[ 12.931166] EDAC sbridge: Seeking for: PCI ID 8086:0eaa :[ 12.931174] EDAC sbridge: Seeking for: PCI ID 8086:0eaa :[ 12.931182] EDAC sbridge: Seeking for: PCI ID 8086:0eaa :[ 12.931190] EDAC sbridge: Seeking for: PCI ID 8086:0eaa :[ 12.931193] EDAC sbridge: Seeking for: PCI ID 8086:0eab :[ 12.931200] EDAC sbridge: Seeking for: PCI ID 8086:0eab :[ 12.931208] EDAC sbridge: Seeking for: PCI ID 8086:0eab :[ 12.931216] EDAC sbridge: Seeking for: PCI ID 8086:0eab :[ 12.931230] EDAC sbridge: Seeking for: PCI ID 8086:0eab :[ 12.931233] EDAC sbridge: Seeking for: PCI ID 8086:0eac :[ 12.931241] EDAC sbridge: Seeking for: PCI ID 8086:0eac :[ 12.931249] EDAC sbridge: Seeking for: PCI ID 8086:0eac :[ 12.931257] EDAC sbridge: Seeking for: PCI ID 8086:0eac :[ 12.931265] EDAC sbridge: Seeking for: PCI ID 8086:0eac :[ 12.931268] EDAC sbridge: Seeking for: PCI ID 8086:0ead :[ 12.931275] EDAC sbridge: Seeking for: PCI ID 8086:0ead :[ 12.931284] EDAC sbridge: Seeking for: PCI ID 8086:0ead :[ 12.931292] EDAC sbridge: Seeking for: PCI ID 8086:0ead :[ 12.931300] EDAC sbridge: Seeking for: PCI ID 8086:0ead :[ 12.931302] EDAC sbridge: Seeking for: PCI ID 8086:0ec8 :[ 12.931311] EDAC sbridge: Seeking for: PCI ID 8086:0ec8 :[ 12.931320] EDAC sbridge: Seeking for: PCI ID 8086:0ec8 :[ 12.931328] EDAC sbridge: Seeking for: PCI ID 8086:0ec8 :[ 12.931336] EDAC sbridge: Seeking for: PCI ID 8086:0ec8 :[ 12.931338] EDAC sbridge: Seeking for: PCI ID 8086:0ec9 :[ 12.931346] EDAC sbridge: Seeking for: PCI ID 8086:0ec9 :[ 12.931354] EDAC sbridge: Seeking for: PCI ID 8086:0ec9 :[ 12.931362] EDAC sbridge: Seeking for: PCI ID 8086:0ec9 :[ 12.931371] EDAC sbridge: Seeking for: PCI ID 8086:0ec9 :[ 12.931373] EDAC sbridge: Seeking for: PCI ID 8086:0eca :[ 12.931381] EDAC sbridge: Seeking for: PCI ID 8086:0eca :[ 12.931388] EDAC sbridge: Seeking for: PCI ID 8086:0eca :[ 12.931396] EDAC sbridge: Seeking for: PCI ID 8086:0eca :[ 12.931404] EDAC sbridge: Seeking for: PCI ID 8086:0eca :[ 12.931406] EDAC sbridge: Seeking for: PCI ID 8086:0e60 :[ 12.931432] EDAC sbridge: Seeking for: PCI ID 8086:0e6a :[ 12.931446] EDAC sbridge: Seeking for: PCI ID 8086:0e6b :[ 12.931459] EDAC sbridge: Seeking for: PCI ID 8086:0e6c :[ 12.931486] EDAC sbridge: Seeking for: PCI ID 8086:0e6d :[ 12.931500] EDAC sbridge: Seeking for: PCI ID 8086:0eb8 :[ 12.931515] EDAC sbridge: Seeking for: PCI ID 8086:0ebc :[ 12.932038] EDAC MC0: Giving out device to 'sbridge_edac.c' 'Ivy Bridge Socket#0': DEV 0000:3f:0e.0 :[ 12.932580] EDAC MC1: Giving out device to 'sbridge_edac.c' 'Ivy Bridge Socket#1': DEV 0000:7f:0e.0 :[ 12.936974] EDAC MC2: Giving out device to 'sbridge_edac.c' 'Ivy Bridge Socket#2': DEV 0000:bf:0e.0 :[ 12.939752] EDAC MC3: Giving out device to 'sbridge_edac.c' 'Ivy Bridge Socket#3': DEV 0000:ff:0e.0 :[ 12.939754] EDAC sbridge: Ver: 1.1.1 :[ 13.570693] md/raid1:md125: active with 2 out of 2 mirrors :[ 13.678887] md125: detected capacity change from 0 to 899999072256 :[ 13.907292] Adding 19530748k swap on /dev/sda5. Priority:-1 extents:1 across:19530748k FS :[ 13.967309] XFS (md0): Mounting V4 Filesystem :[ 13.968415] XFS (md126): Mounting V4 Filesystem :[ 14.195696] XFS (md0): Ending clean mount :[ 14.240324] XFS (md125): Mounting V4 Filesystem :[ 14.309061] device-mapper: uevent: version 1.0.3 :[ 14.309310] device-mapper: ioctl: 4.37.1-ioctl (2018-04-03) initialised: dm-devel at redhat.com :[ 15.225567] XFS (md125): Ending clean mount :[ 15.409418] XFS (md126): Ending clean mount :[ 15.786361] RPC: Registered named UNIX socket transport module. :[ 15.786366] RPC: Registered udp transport module. :[ 15.786368] RPC: Registered tcp transport module. :[ 15.786369] RPC: Registered tcp NFSv4.1 backchannel transport module. :[ 15.819376] type=1305 audit(1535484954.832:4): audit_pid=1072 old=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1 :[ 16.712564] Process accounting resumed :[ 17.291335] nf_conntrack version 0.5.0 (65536 buckets, 262144 max) :[ 18.613147] IPv6: ADDRCONF(NETDEV_UP): enp193s0f0: link is not ready :[ 18.725757] IPv6: ADDRCONF(NETDEV_UP): enp193s0f0: link is not ready :[ 18.732037] IPv6: ADDRCONF(NETDEV_UP): enp193s0f1: link is not ready :[ 18.843427] IPv6: ADDRCONF(NETDEV_UP): enp193s0f1: link is not ready :[ 18.913026] IPv6: ADDRCONF(NETDEV_UP): enp193s0f0: link is not ready :[ 18.913366] IPv6: ADDRCONF(NETDEV_UP): enp193s0f1: link is not ready :[ 18.916060] IPv6: ADDRCONF(NETDEV_UP): enp193s0f0: link is not ready :[ 22.784419] igb 0000:c1:00.0 enp193s0f0: igb: enp193s0f0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX :[ 22.784593] IPv6: ADDRCONF(NETDEV_CHANGE): enp193s0f0: link becomes ready :[ 23.241409] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX :[ 23.847136] IPv6: ADDRCONF(NETDEV_CHANGE): enp193s0f1: link becomes ready :[ 29.622173] FS-Cache: Loaded :[ 29.688815] FS-Cache: Netfs 'nfs' registered for caching :[ 29.704139] Key type dns_resolver registered :[ 29.740952] NFS: Registering the id_resolver key type :[ 29.740958] Key type id_resolver registered :[ 29.740959] Key type id_legacy registered :[ 30.673307] ip6_tables: (C) 2000-2006 Netfilter Core Team :[ 30.711260] Ebtables v2.0 registered :[ 8475.790712] perf: interrupt took too long (2569 > 2500), lowering kernel.perf_event_max_sample_rate to 77000 :[10884.100521] perf: interrupt took too long (3240 > 3211), lowering kernel.perf_event_max_sample_rate to 61000 :[14442.537676] perf: interrupt took too long (4090 > 4050), lowering kernel.perf_event_max_sample_rate to 48000 :[19258.232327] perf: interrupt took too long (5123 > 5112), lowering kernel.perf_event_max_sample_rate to 39000 :[25478.993669] perf: interrupt took too long (6406 > 6403), lowering kernel.perf_event_max_sample_rate to 31000 :[36765.253517] perf: interrupt took too long (8022 > 8007), lowering kernel.perf_event_max_sample_rate to 24000 :[56544.781126] perf: interrupt took too long (10031 > 10027), lowering kernel.perf_event_max_sample_rate to 19000 :[62098.795370] Process accounting resumed :[131376.313379] Process accounting resumed :[178609.054001] nfs: server uranus not responding, timed out :[178789.534339] nfs: server uranus not responding, timed out :[178970.014707] nfs: server uranus not responding, timed out :[179150.495104] nfs: server uranus not responding, timed out :[179330.975574] nfs: server uranus not responding, timed out :[179511.456046] nfs: server uranus not responding, timed out :[179691.936552] nfs: server uranus not responding, timed out :[179872.417028] nfs: server uranus not responding, timed out :[180052.897525] nfs: server uranus not responding, timed out :[180233.378086] nfs: server uranus not responding, timed out :[221455.205385] Process accounting resumed :[235307.614882] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Down :[235309.315158] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 10 Mbps Full Duplex, Flow Control: RX/TX :[235312.059894] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Down :[235313.759224] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 10 Mbps Full Duplex, Flow Control: RX/TX :[235314.463900] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Down :[235316.160158] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 10 Mbps Full Duplex, Flow Control: RX/TX :[235344.049969] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Down :[235347.576249] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX :[235380.390054] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Down :[235405.797371] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX :[303178.575714] Process accounting resumed :[379463.673156] md: data-check of RAID array md125 :[379469.878565] md: data-check of RAID array md0 :[379476.004208] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[379482.146223] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[385825.554816] md: md125: data-check done. :[385825.570887] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[385825.570903] md: data-check of RAID array md126 :[385832.512392] md: md126: data-check done. :[385832.524171] md: data-check of RAID array md127 :[386178.584697] md: md127: data-check done. :[403493.687041] Process accounting resumed :[409764.444414] md: md0: data-check done. :[481122.279383] Process accounting resumed :[564992.336712] Process accounting resumed :[664909.913022] Process accounting resumed :[749185.329403] Process accounting resumed :[832403.489618] Process accounting resumed :[921305.340538] Process accounting resumed :[984264.958998] md: data-check of RAID array md125 :[984271.043293] md: data-check of RAID array md0 :[984277.102530] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[984285.032687] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[990677.188762] md: md125: data-check done. :[990677.205737] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[990677.205739] md: data-check of RAID array md126 :[990686.161896] md: md126: data-check done. :[990686.173938] md: data-check of RAID array md127 :[991088.284754] md: md127: data-check done. :[1001183.389970] Process accounting resumed :[1012429.096998] md: md0: data-check done. :[1050534.967008] perf: interrupt took too long (12541 > 12538), lowering kernel.perf_event_max_sample_rate to 15000 :[1084911.394691] Process accounting resumed :[1177619.692106] sd 0:0:2:0: attempting task abort! scmd(ffff93a2c6368380) :[1177619.692121] sd 0:0:2:0: [sdc] CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00 :[1177619.692129] scsi target0:0:2: handle(0x000b), sas_address(0x4433221103000000), phy(3) :[1177619.692135] scsi target0:0:2: enclosure logical id(0x500605b00a2392e0), slot(1) :[1177619.692139] scsi target0:0:2: enclosure level(0x0000), connector name( ) :[1177619.876477] sd 0:0:2:0: task abort: SUCCESS scmd(ffff93a2c6368380) :[1177620.971424] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971432] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971437] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971443] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971447] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971453] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971456] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971460] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971463] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971467] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971471] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971474] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971478] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971481] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971487] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971491] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971496] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971500] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971504] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971508] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971510] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971516] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971518] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971524] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971528] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971534] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971537] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971542] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971546] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971564] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971568] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1177620.971573] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1187924.959093] Process accounting resumed :[1251799.745540] sd 0:0:2:0: attempting task abort! scmd(ffff940358bff9c0) :[1251799.745555] sd 0:0:2:0: [sdc] CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00 :[1251799.745565] scsi target0:0:2: handle(0x000b), sas_address(0x4433221103000000), phy(3) :[1251799.745571] scsi target0:0:2: enclosure logical id(0x500605b00a2392e0), slot(1) :[1251799.745575] scsi target0:0:2: enclosure level(0x0000), connector name( ) :[1251799.930659] sd 0:0:2:0: task abort: SUCCESS scmd(ffff940358bff9c0) :[1251800.878351] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878363] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878369] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878374] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878380] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878386] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878391] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878396] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878402] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878407] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878413] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878418] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878424] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878429] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878434] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878440] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878445] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878451] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878456] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878460] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878465] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878471] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878475] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878482] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878487] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878493] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878498] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878503] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878508] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878515] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878520] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1251800.878526] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1258072.441663] Process accounting resumed :[1342435.577924] Process accounting resumed :[1430291.960988] Process accounting resumed :[1511395.006664] Process accounting resumed :[1522464.717746] sd 0:0:0:0: attempting task abort! scmd(ffff93a456b801c0) :[1522464.717763] sd 0:0:0:0: [sda] CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00 :[1522464.717772] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[1522464.717778] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[1522464.717783] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[1522464.902818] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93a456b801c0) :[1522465.941427] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941437] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941446] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941450] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941455] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941460] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941466] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941472] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941488] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941491] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941496] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941499] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941502] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941504] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941507] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941510] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941513] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941516] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941518] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941521] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941524] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941527] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941529] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941532] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941535] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941551] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941557] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941561] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941565] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941570] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941574] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522465.941578] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924562] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924571] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924576] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924580] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924584] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924588] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924592] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924596] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924600] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924604] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924608] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924611] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924617] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924619] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924625] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924629] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924632] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924636] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924640] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924644] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924648] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924652] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924656] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924659] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924663] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924667] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924671] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924674] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924678] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924682] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924686] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522466.924693] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898501] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898510] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898514] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898518] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898522] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898526] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898530] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898535] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898540] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898543] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898550] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898554] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898558] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898561] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898564] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898568] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898572] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898575] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898579] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898582] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898586] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898589] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898593] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898596] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898600] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898603] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898607] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898611] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898615] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898617] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898622] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1522467.898629] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1589066.865315] md: data-check of RAID array md125 :[1589072.951867] md: data-check of RAID array md0 :[1589078.988948] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[1589086.981297] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[1595473.613939] md: md125: data-check done. :[1595473.630946] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[1595473.630965] md: data-check of RAID array md126 :[1595480.996907] md: md126: data-check done. :[1595481.008307] md: data-check of RAID array md127 :[1595932.003970] md: md127: data-check done. :[1615945.789151] Process accounting resumed :[1617105.860079] md: md0: data-check done. :[1666903.876336] sd 0:0:2:0: attempting task abort! scmd(ffff93c7c9717480) :[1666903.876354] sd 0:0:2:0: [sdc] CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00 :[1666903.876363] scsi target0:0:2: handle(0x000b), sas_address(0x4433221103000000), phy(3) :[1666903.876368] scsi target0:0:2: enclosure logical id(0x500605b00a2392e0), slot(1) :[1666903.876372] scsi target0:0:2: enclosure level(0x0000), connector name( ) :[1666904.061175] sd 0:0:2:0: task abort: SUCCESS scmd(ffff93c7c9717480) :[1666904.984507] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984519] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984526] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984531] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984536] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984542] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984547] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984552] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984557] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984563] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984568] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984573] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984579] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984584] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984590] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984595] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984600] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984605] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984611] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984616] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984621] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984626] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984632] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984637] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984642] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984647] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984653] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984658] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984663] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984668] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984673] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1666904.984678] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[1689100.925983] Process accounting resumed :[1773115.340533] Process accounting resumed :[1862930.843217] Process accounting resumed :[1945601.368180] Process accounting resumed :[2045988.181943] Process accounting resumed :[2121331.646899] Process accounting resumed :[2193868.133020] md: data-check of RAID array md125 :[2193874.217104] md: data-check of RAID array md0 :[2193880.273181] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[2193888.199701] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[2200212.190285] md: md125: data-check done. :[2200212.207033] md: data-check of RAID array md126 :[2200212.207050] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[2200218.580339] md: md126: data-check done. :[2200218.592368] md: data-check of RAID array md127 :[2200648.058198] md: md127: data-check done. :[2216190.542432] Process accounting resumed :[2222024.398888] md: md0: data-check done. :[2293291.346423] Process accounting resumed :[2394521.252041] Process accounting resumed :[2477867.462632] Process accounting resumed :[2561935.680908] Process accounting resumed :[2646289.488409] Process accounting resumed :[2740658.779218] Process accounting resumed :[2798669.685277] md: data-check of RAID array md125 :[2798675.764050] md: data-check of RAID array md0 :[2798681.798331] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[2798687.840606] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[2805039.906030] md: md125: data-check done. :[2805039.922616] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[2805039.922618] md: data-check of RAID array md126 :[2805048.039674] md: md126: data-check done. :[2805048.051104] md: data-check of RAID array md127 :[2805658.334400] md: md127: data-check done. :[2814998.684917] Process accounting resumed :[2826718.765083] md: md0: data-check done. :[2914212.818239] Process accounting resumed :[2946261.213760] Process accounting paused :[2946821.746347] Process accounting resumed :[2947559.873750] SELinux: 2048 avtab hash slots, 106487 rules. :[2947559.953237] SELinux: 2048 avtab hash slots, 106487 rules. :[2947559.991945] SELinux: 8 users, 14 roles, 5009 types, 310 bools, 1 sens, 1024 cats :[2947559.991949] SELinux: 97 classes, 106487 rules :[2947658.050258] This updated microcode supersedes microcode provided by Red Hat with :the CVE-2017-5715 (?Spectre?) CPU branch injection vulnerability :mitigation. :Historically, Red Hat has provided updated microcode, developed by our :microprocessor partners, as a customer convenience. Red Hat had :temporarily suspended this practice while microcode stabilized. Red :Hat is once again providing an updated Intel microcode package :(microcode_ctl) and AMD microcode package (linux-firmware) to customers :in order to simplify deployment processes and minimize downtime. We?ll :continue to update these microcode packages as necessary. Please :contact your hardware vendor to determine whether more recent :BIOS/firmware updates are recommended because additional improvements :may be available. :This kbase https://access.redhat.com/articles/3436091 includes a table :that maps Intel and AMD CPU processor code family names to updated :Intel and AMD microcode package versions. :[2998532.730370] Process accounting resumed :[3083804.065972] Process accounting resumed :[3169781.627115] Process accounting resumed :[3243468.274980] Process accounting resumed :[3325229.411669] Process accounting resumed :[3403470.661987] md: data-check of RAID array md125 :[3403476.851804] md: data-check of RAID array md0 :[3403482.884685] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[3403489.831173] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[3409833.987600] md: md125: data-check done. :[3409834.004281] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[3409834.004326] md: data-check of RAID array md126 :[3409840.913213] md: md126: data-check done. :[3409840.924696] md: data-check of RAID array md127 :[3410381.999518] md: md127: data-check done. :[3415340.259591] Process accounting resumed :[3431524.690198] md: md0: data-check done. :[3500238.228489] Process accounting resumed :[3604184.021260] Process accounting resumed :[3679928.199329] Process accounting resumed :[3760543.576853] Process accounting resumed :[3852624.426289] Process accounting resumed :[3934997.720318] Process accounting resumed :[4008272.246482] md: data-check of RAID array md125 :[4008278.454168] md: data-check of RAID array md0 :[4008284.487623] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[4008292.163870] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[4014658.990718] md: md125: data-check done. :[4014659.006913] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[4014659.006920] md: data-check of RAID array md126 :[4014666.039657] md: md126: data-check done. :[4014666.051113] md: data-check of RAID array md127 :[4015336.969949] md: md127: data-check done. :[4017885.865683] Process accounting resumed :[4036192.313420] md: md0: data-check done. :[4108064.165477] Process accounting resumed :[4192363.428348] Process accounting resumed :[4281974.806472] Process accounting resumed :[4368275.877638] Process accounting resumed :[4459196.422150] Process accounting resumed :[4541105.568465] Process accounting resumed :[4613073.809161] md: data-check of RAID array md125 :[4613079.863469] md: data-check of RAID array md0 :[4613085.897205] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[4613094.885637] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[4619406.483683] md: md125: data-check done. :[4619406.500102] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[4619406.500105] md: data-check of RAID array md126 :[4619412.957111] md: md126: data-check done. :[4619412.969381] md: data-check of RAID array md127 :[4619965.927555] md: md127: data-check done. :[4633462.425960] Process accounting resumed :[4641451.897158] md: md0: data-check done. :[4693876.929786] sd 0:0:2:0: attempting task abort! scmd(ffff94014fb01c00) :[4693876.929797] sd 0:0:2:0: [sdc] CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00 :[4693876.929802] scsi target0:0:2: handle(0x000b), sas_address(0x4433221103000000), phy(3) :[4693876.929806] scsi target0:0:2: enclosure logical id(0x500605b00a2392e0), slot(1) :[4693876.929808] scsi target0:0:2: enclosure level(0x0000), connector name( ) :[4693877.114210] sd 0:0:2:0: task abort: SUCCESS scmd(ffff94014fb01c00) :[4693877.687849] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687859] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687864] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687869] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687874] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687879] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687884] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687889] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687894] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687908] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687913] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687918] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687922] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687927] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687932] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687937] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687942] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687946] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687951] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687956] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687961] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687966] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687971] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687976] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687980] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687985] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687990] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687995] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.687999] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.688004] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.688009] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693877.688013] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437844] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437852] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437856] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437861] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437865] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437869] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437873] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437877] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437882] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437886] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437891] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437895] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437900] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437905] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437910] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437915] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437919] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437924] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437929] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437934] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437938] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437942] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437947] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437952] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437957] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437961] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437966] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437971] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437976] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437981] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437985] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.437989] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[4693878.770559] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770571] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770575] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770580] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770584] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770588] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770593] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770598] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770603] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770608] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770613] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770618] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770623] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770628] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770632] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770637] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770643] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770648] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770653] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770658] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770663] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770668] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770673] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770678] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770684] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770689] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770694] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770698] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770703] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770709] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770714] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4693878.770718] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[4709308.945033] Process accounting resumed :[4806462.713983] Process accounting resumed :[4897216.206599] Process accounting resumed :[4979105.247864] Process accounting resumed :[5056863.055510] Process accounting resumed :[5156901.978154] Process accounting resumed :[5217874.837402] md: data-check of RAID array md125 :[5217881.035188] md: data-check of RAID array md0 :[5217887.069988] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[5217896.253642] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[5224244.568175] md: md125: data-check done. :[5224244.584549] md: data-check of RAID array md126 :[5224244.584568] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[5224251.560561] md: md126: data-check done. :[5224251.572045] md: data-check of RAID array md127 :[5224844.948685] md: md127: data-check done. :[5233207.584317] Process accounting resumed :[5246327.012530] md: md0: data-check done. :[5325142.475732] sd 0:0:2:0: attempting task abort! scmd(ffff94060e7eebc0) :[5325142.475748] sd 0:0:2:0: [sdc] CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00 :[5325142.475756] scsi target0:0:2: handle(0x000b), sas_address(0x4433221103000000), phy(3) :[5325142.475763] scsi target0:0:2: enclosure logical id(0x500605b00a2392e0), slot(1) :[5325142.475768] scsi target0:0:2: enclosure level(0x0000), connector name( ) :[5325142.660230] sd 0:0:2:0: task abort: SUCCESS scmd(ffff94060e7eebc0) :[5325614.520044] Process accounting resumed :[5362118.568785] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee0fc0) :[5362118.568805] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 05 4b e4 e0 00 00 08 00 :[5362118.568814] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.568819] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.568823] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.793133] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee0fc0) :[5362118.793166] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff0e00) :[5362118.793186] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 05 4b e4 f0 00 00 10 00 :[5362118.793192] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.793197] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.793201] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.793357] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff0e00) :[5362118.793371] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff0540) :[5362118.793377] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 05 4b e5 10 00 00 08 00 :[5362118.793382] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.793386] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.793390] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.793448] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff0540) :[5362118.793461] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff0000) :[5362118.793467] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 05 4b e5 20 00 00 08 00 :[5362118.793472] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.793476] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.793480] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.793538] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff0000) :[5362118.793550] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff1500) :[5362118.793556] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 05 4b e5 40 00 00 10 00 :[5362118.793560] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.793564] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.793568] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.793627] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff1500) :[5362118.793639] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff0700) :[5362118.793645] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ac 28 00 00 10 00 :[5362118.793649] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.793653] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.793657] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.793740] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff0700) :[5362118.793755] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff2840) :[5362118.793760] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ac 40 00 00 10 00 :[5362118.793765] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.793769] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.793773] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.793831] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff2840) :[5362118.793844] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff3100) :[5362118.793850] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ac 70 00 00 08 00 :[5362118.793855] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.793859] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.793863] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.793921] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff3100) :[5362118.793934] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff39c0) :[5362118.793939] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ac 80 00 00 10 00 :[5362118.793944] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.793948] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.793952] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.794010] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff39c0) :[5362118.794023] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff32c0) :[5362118.794030] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ac a8 00 00 08 00 :[5362118.794034] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.794038] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.794042] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.794100] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff32c0) :[5362118.794112] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff2d80) :[5362118.794118] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ac c0 00 00 08 00 :[5362118.794122] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.794126] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.794130] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.794188] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff2d80) :[5362118.794200] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff16c0) :[5362118.794206] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ad 38 00 00 10 00 :[5362118.794211] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.794215] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.794219] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.794277] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff16c0) :[5362118.794289] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff1a40) :[5362118.794294] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ad 58 00 00 08 00 :[5362118.794299] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.794303] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.794307] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.794365] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff1a40) :[5362118.794377] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff08c0) :[5362118.794383] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ad 68 00 00 08 00 :[5362118.794388] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.794392] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.794396] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.794454] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff08c0) :[5362118.794466] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff2140) :[5362118.794471] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ad 78 00 00 08 00 :[5362118.794476] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.794480] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.794484] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.794542] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff2140) :[5362118.794554] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff3b80) :[5362118.794560] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ad 90 00 00 08 00 :[5362118.794565] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.794569] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.794573] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.794631] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff3b80) :[5362118.794643] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff3d40) :[5362118.794648] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ad a8 00 00 10 00 :[5362118.794653] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.794657] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.794661] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.794732] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff3d40) :[5362118.794747] sd 0:0:0:0: attempting task abort! scmd(ffff93ee3bff0380) :[5362118.794752] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ad c8 00 00 08 00 :[5362118.794757] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.794761] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.794765] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.794823] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93ee3bff0380) :[5362118.794841] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee2840) :[5362118.794847] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ad d8 00 00 08 00 :[5362118.794852] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.794856] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.794860] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.794918] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee2840) :[5362118.794936] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee1dc0) :[5362118.794942] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ae 60 00 00 08 00 :[5362118.794947] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.794951] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.794955] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.795013] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee1dc0) :[5362118.795032] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee2140) :[5362118.795037] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ae 70 00 00 10 00 :[5362118.795042] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.795046] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.795050] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.795108] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee2140) :[5362118.795120] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee39c0) :[5362118.795126] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ae 98 00 00 08 00 :[5362118.795130] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.795134] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.795138] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.795196] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee39c0) :[5362118.795208] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee0c40) :[5362118.795213] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ae c0 00 00 08 00 :[5362118.795218] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.795222] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.795226] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.795284] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee0c40) :[5362118.795297] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee0700) :[5362118.795303] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ae e0 00 00 08 00 :[5362118.795307] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.795312] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.795315] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.795373] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee0700) :[5362118.795386] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee1500) :[5362118.795391] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c ae f8 00 00 10 00 :[5362118.795396] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.795400] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.795404] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.795462] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee1500) :[5362118.795473] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee0380) :[5362118.795478] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c af 88 00 00 08 00 :[5362118.795483] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.795487] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.795491] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.795549] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee0380) :[5362118.795563] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee16c0) :[5362118.795568] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c af 98 00 00 08 00 :[5362118.795573] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.795577] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.795581] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.795639] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee16c0) :[5362118.795650] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee3100) :[5362118.795656] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c af b0 00 00 10 00 :[5362118.795661] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.795665] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.795669] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.795738] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee3100) :[5362118.795751] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee1880) :[5362118.795757] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c af d0 00 00 08 00 :[5362118.795762] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.795766] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.795770] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.795828] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee1880) :[5362118.795840] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee1f80) :[5362118.795845] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c af e0 00 00 08 00 :[5362118.795850] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.795854] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.795858] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.795917] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee1f80) :[5362118.795928] sd 0:0:0:0: attempting task abort! scmd(ffff93c47fee3640) :[5362118.795934] sd 0:0:0:0: [sda] CDB: Write(10) 2a 00 00 0c af f8 00 00 08 00 :[5362118.795938] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5362118.795943] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5362118.795946] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5362118.796004] sd 0:0:0:0: task abort: SUCCESS scmd(ffff93c47fee3640) :[5362119.783276] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783287] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783293] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783298] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783304] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783309] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783314] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783322] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783324] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783330] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783334] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783339] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783343] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783347] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783351] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783356] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783362] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783368] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783373] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783379] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783385] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783387] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783392] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783397] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783399] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783403] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783408] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783412] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783415] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783418] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783426] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5362119.783432] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5415031.744029] Process accounting resumed :[5467568.936253] sd 0:0:0:0: attempting task abort! scmd(ffff9402e5f401c0) :[5467568.936269] sd 0:0:0:0: [sda] CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00 :[5467568.936277] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[5467568.936281] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[5467568.936285] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[5467569.121404] sd 0:0:0:0: task abort: SUCCESS scmd(ffff9402e5f401c0) :[5467570.223258] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223268] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223272] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223276] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223280] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223284] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223288] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223291] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223296] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223300] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223304] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223308] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223321] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223326] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223330] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223335] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223339] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223343] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223348] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223352] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223356] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223361] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223365] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223370] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223374] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223379] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223383] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223387] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223392] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223396] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223400] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467570.223405] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277893] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277901] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277906] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277911] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277916] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277921] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277925] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277930] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277935] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277939] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277944] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277949] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277954] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277958] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277963] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277968] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277973] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277977] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277982] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277986] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277990] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277995] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.277999] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.278004] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.278009] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.278014] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.278018] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.278023] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.278028] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.278033] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.278037] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467571.278041] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278227] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278236] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278241] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278245] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278250] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278254] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278259] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278264] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278268] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278273] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278278] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278284] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278289] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278293] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278298] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278303] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278308] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278313] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278318] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278323] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278328] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278333] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278337] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278342] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278346] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278351] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278356] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278361] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278366] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278371] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278376] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5467572.278381] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[5506407.082041] Process accounting resumed :[5579823.514542] Process accounting resumed :[5663666.093723] Process accounting resumed :[5749222.832476] Process accounting resumed :[5822676.835714] md: data-check of RAID array md125 :[5822682.923499] md: data-check of RAID array md0 :[5822688.956499] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[5822696.915298] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[5829012.015087] md: md125: data-check done. :[5829012.031604] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[5829012.031607] md: data-check of RAID array md126 :[5829018.118634] md: md126: data-check done. :[5829018.130264] md: data-check of RAID array md127 :[5829607.437968] md: md127: data-check done. :[5846206.572984] Process accounting resumed :[5850951.835481] md: md0: data-check done. :[5933506.683971] Process accounting resumed :[5966016.822333] nfs: server uranus not responding, timed out :[5966197.303427] nfs: server uranus not responding, timed out :[5966377.784493] nfs: server uranus not responding, timed out :[5966558.265346] nfs: server uranus not responding, timed out :[6017719.581311] Process accounting resumed :[6083343.221585] sd 0:0:2:0: attempting task abort! scmd(ffff940140b44a80) :[6083343.221610] sd 0:0:2:0: [sdc] CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00 :[6083343.221619] scsi target0:0:2: handle(0x000b), sas_address(0x4433221103000000), phy(3) :[6083343.221625] scsi target0:0:2: enclosure logical id(0x500605b00a2392e0), slot(1) :[6083343.221629] scsi target0:0:2: enclosure level(0x0000), connector name( ) :[6083343.406569] sd 0:0:2:0: task abort: SUCCESS scmd(ffff940140b44a80) :[6083344.539443] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539456] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539462] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539468] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539473] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539479] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539484] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539490] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539495] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539501] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539506] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539512] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539517] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539523] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539528] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539534] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539540] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539545] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539550] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539556] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539561] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539567] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539573] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539578] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539583] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539589] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539594] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539600] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539606] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539611] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539617] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6083344.539622] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[6115920.123692] Process accounting resumed :[6183190.236502] Process accounting resumed :[6254980.647344] sd 0:0:2:0: attempting task abort! scmd(ffff9408fe2c4fc0) :[6254980.647359] sd 0:0:2:0: [sdc] CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00 :[6254980.647369] scsi target0:0:2: handle(0x000b), sas_address(0x4433221103000000), phy(3) :[6254980.647375] scsi target0:0:2: enclosure logical id(0x500605b00a2392e0), slot(1) :[6254980.647379] scsi target0:0:2: enclosure level(0x0000), connector name( ) :[6254980.832599] sd 0:0:2:0: task abort: SUCCESS scmd(ffff9408fe2c4fc0) :[6282004.649063] Process accounting resumed :[6365231.999995] Process accounting resumed :[6431078.058553] md: data-check of RAID array md125 :[6431084.144620] md: data-check of RAID array md0 :[6431090.177892] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[6431096.550774] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[6437464.041237] md: md125: data-check done. :[6437464.057569] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[6437464.057575] md: data-check of RAID array md126 :[6437472.131108] md: md126: data-check done. :[6437472.143020] md: data-check of RAID array md127 :[6438156.268641] md: md127: data-check done. :[6450805.598941] Process accounting resumed :[6459816.242301] md: md0: data-check done. :[6532690.267022] Process accounting resumed :[6614364.635550] Process accounting resumed :[6702981.493267] Process accounting resumed :[6796956.659418] Process accounting resumed :[6882170.191482] Process accounting resumed :[6943578.008046] SELinux: 2048 avtab hash slots, 111641 rules. :[6943578.068536] SELinux: 2048 avtab hash slots, 111641 rules. :[6943578.096288] SELinux: 8 users, 14 roles, 5026 types, 315 bools, 1 sens, 1024 cats :[6943578.096293] SELinux: 129 classes, 111641 rules :[6943608.801304] This updated microcode supersedes microcode provided by Red Hat with :the CVE-2017-5715 (?Spectre?) CPU branch injection vulnerability :mitigation. :Historically, Red Hat has provided updated microcode, developed by our :microprocessor partners, as a customer convenience. Red Hat had :temporarily suspended this practice while microcode stabilized. Red :Hat is once again providing an updated Intel microcode package :(microcode_ctl) and AMD microcode package (linux-firmware) to customers :in order to simplify deployment processes and minimize downtime. We?ll :continue to update these microcode packages as necessary. Please :contact your hardware vendor to determine whether more recent :BIOS/firmware updates are recommended because additional improvements :may be available. :This kbase https://access.redhat.com/articles/3436091 includes a table :that maps Intel and AMD CPU processor code family names to updated :Intel and AMD microcode package versions. :[6943851.594110] type=1305 audit(1542428774.136:112650): audit_pid=0 old=1072 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1 :[6943851.600701] type=1130 audit(1542428774.142:112651): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=auditd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' :[6943851.600753] type=1131 audit(1542428774.142:112652): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=auditd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' :[6943852.622896] type=1305 audit(1542428775.165:112653): audit_enabled=1 old=1 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1 :[6943852.622922] type=1305 audit(1542428775.165:112654): audit_pid=14923 old=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1 :[6950166.595291] SELinux: 2048 avtab hash slots, 111624 rules. :[6950166.686506] SELinux: 2048 avtab hash slots, 111624 rules. :[6950166.737992] SELinux: 8 users, 14 roles, 5025 types, 315 bools, 1 sens, 1024 cats :[6950166.737997] SELinux: 129 classes, 111624 rules :[6961391.870101] Process accounting resumed :[6989928.295426] lockd: server gaia not responding, still trying :[6990232.633089] lockd: server gaia OK :[7035879.465641] md: data-check of RAID array md125 :[7035885.560839] md: data-check of RAID array md0 :[7035891.620502] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[7035898.717743] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[7042240.405113] md: md125: data-check done. :[7042240.421448] md: data-check of RAID array md126 :[7042240.421478] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[7042247.266978] md: md126: data-check done. :[7042247.278410] md: data-check of RAID array md127 :[7042891.245971] md: md127: data-check done. :[7048474.306072] Process accounting resumed :[7064726.415154] md: md0: data-check done. :[7123473.804069] sd 0:0:0:0: attempting task abort! scmd(ffff940130e7a680) :[7123473.804085] sd 0:0:0:0: [sda] CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00 :[7123473.804094] scsi target0:0:0: handle(0x000a), sas_address(0x4433221102000000), phy(2) :[7123473.804101] scsi target0:0:0: enclosure logical id(0x500605b00a2392e0), slot(0) :[7123473.804105] scsi target0:0:0: enclosure level(0x0000), connector name( ) :[7123473.988946] sd 0:0:0:0: task abort: SUCCESS scmd(ffff940130e7a680) :[7123475.048961] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.048974] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.048979] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.048984] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.048989] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.048995] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049000] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049015] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049020] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049025] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049031] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049036] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049042] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049047] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049052] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049058] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049062] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049067] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049072] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049078] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049083] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049089] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049094] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049099] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049104] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049109] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049115] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049120] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049125] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049130] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049136] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123475.049141] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007888] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007898] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007903] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007908] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007913] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007918] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007923] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007928] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007933] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007938] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007944] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007949] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007954] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007960] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007965] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007970] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007976] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007981] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007986] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007991] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.007996] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.008001] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.008006] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.008012] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.008017] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.008022] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.008027] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.008032] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.008037] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.008043] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.008048] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7123476.008054] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7144398.981243] Process accounting resumed :[7198398.145403] lockd: server gaia not responding, still trying :[7198422.273423] lockd: server gaia not responding, still trying :[7234026.804446] Process accounting resumed :[7315580.821726] Process accounting resumed :[7395158.802998] Process accounting resumed :[7413620.621889] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Down :[7413622.453148] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 10 Mbps Full Duplex, Flow Control: RX/TX :[7413649.426920] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Down :[7413652.925172] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX :[7413685.857960] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Down :[7413708.764253] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX :[7434334.912856] lockd: server gaia OK :[7434485.885508] lockd: server gaia OK :[7496056.239984] Process accounting resumed :[7565778.224380] Process accounting resumed :[7640680.070410] md: data-check of RAID array md125 :[7640686.286107] md: data-check of RAID array md0 :[7640692.320343] md: delaying data-check of md126 until md125 has finished (they share one or more physical units) :[7640700.136487] md: delaying data-check of md127 until md125 has finished (they share one or more physical units) :[7647021.399852] md: md125: data-check done. :[7647021.416050] md: data-check of RAID array md126 :[7647021.416053] md: delaying data-check of md127 until md126 has finished (they share one or more physical units) :[7647028.006565] md: md126: data-check done. :[7647028.018468] md: data-check of RAID array md127 :[7647614.406281] md: md127: data-check done. :[7666070.336488] Process accounting resumed :[7669309.302447] md: md0: data-check done. :[7740237.376395] Process accounting resumed :[7823888.755811] Process accounting resumed :[7893318.562760] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Down :[7893320.261032] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 10 Mbps Full Duplex, Flow Control: RX/TX :[7893347.275831] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Down :[7893350.495081] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX :[7893383.270852] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Down :[7893417.476189] igb 0000:c1:00.1 enp193s0f1: igb: enp193s0f1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX :[7920027.440241] Process accounting resumed :[7944282.095791] sd 0:0:2:0: attempting task abort! scmd(ffff940358a07640) :[7944282.095806] sd 0:0:2:0: [sdc] CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00 :[7944282.095816] scsi target0:0:2: handle(0x000b), sas_address(0x4433221103000000), phy(3) :[7944282.095821] scsi target0:0:2: enclosure logical id(0x500605b00a2392e0), slot(1) :[7944282.095826] scsi target0:0:2: enclosure level(0x0000), connector name( ) :[7944282.280644] sd 0:0:2:0: task abort: SUCCESS scmd(ffff940358a07640) :[7944283.258322] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258333] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258338] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258356] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258360] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258365] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258369] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258373] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258377] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258382] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258386] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258391] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258395] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258400] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258404] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258416] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258424] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258429] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258433] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258437] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258441] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258446] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258450] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258454] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258460] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258465] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258469] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258474] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258479] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258483] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258488] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944283.258499] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.255959] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.910951] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.910974] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.910979] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.910984] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.910989] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.910994] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.910998] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911004] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911018] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911023] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911028] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911033] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911038] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911043] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911048] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911054] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911060] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911065] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911070] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911075] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911080] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911085] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911090] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911096] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911101] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911106] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911110] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911114] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911119] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911124] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911129] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944284.911133] mpt3sas_cm0: log_info(0x31110e03): originator(PL), code(0x11), sub_code(0x0e03) :[7944285.236787] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236796] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236801] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236805] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236809] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236813] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236818] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236822] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236827] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236831] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236837] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236841] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236845] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236850] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236855] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236860] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236865] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236870] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236876] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236881] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236886] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236891] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236896] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236901] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236906] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236911] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236917] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236922] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236927] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236932] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236937] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[7944285.236942] mpt3sas_cm0: log_info(0x31120b10): originator(PL), code(0x12), sub_code(0x0b10) :[8000693.286565] Process accounting resumed :[8066421.244603] mce: [Hardware Error]: Machine check events logged :[8066421.244635] mce: [Hardware Error]: Machine check events logged :[8066421.244697] EDAC sbridge MC2: HANDLING MCE MEMORY ERROR :[8066421.244699] EDAC sbridge MC2: CPU 16: Machine Check Event: 0 Bank 11: 8c00004e000800c2 :[8066421.244700] EDAC sbridge MC2: TSC 0 :[8066421.244702] EDAC sbridge MC2: ADDR 5d7be58000 :[8066421.244703] EDAC sbridge MC2: MISC 1229400040005c8c :[8066421.244704] EDAC sbridge MC2: PROCESSOR 0:306e4 TIME 1543551341 SOCKET 2 APIC 40 :[8066421.244706] EDAC sbridge MC2: HANDLING MCE MEMORY ERROR :[8066421.244707] EDAC sbridge MC2: CPU 16: Machine Check Event: 0 Bank 11: 8c00004e000800c2 :[8066421.244708] EDAC sbridge MC2: TSC 0 :[8066421.244709] EDAC sbridge MC2: ADDR 5d7be59000 :[8066421.244710] EDAC sbridge MC2: MISC 1229404000401c8c :[8066421.244711] EDAC sbridge MC2: PROCESSOR 0:306e4 TIME 1543551341 SOCKET 2 APIC 40 :[8066421.490370] EDAC MC2: 1 CE memory scrubbing error on CPU_SrcID#2_Ha#0_Chan#0_DIMM#1 (channel:0 slot:1 page:0x5d7be58 offset:0x0 grain:32 syndrome:0x0 - area:DRAM err_code:0008:00c2 socket:2 ha:0 channel_mask:1 rank:5) :[8066421.490377] EDAC MC2: 1 CE memory scrubbing error on CPU_SrcID#2_Ha#0_Chan#0_DIMM#1 (channel:0 slot:1 page:0x5d7be59 offset:0x0 grain:32 syndrome:0x0 - area:DRAM err_code:0008:00c2 socket:2 ha:0 channel_mask:1 rank:5) machineid: :systemd=20b27ad50adc448f8f4f601d0f85c4c5 :sosreport_uploader-dmidecode=abfbd49a756d50816ec770f6127cd664a4d806d5fe878b8afded23cbb7892509 not-reportable: :The kernel log indicates that hardware errors were detected. :This is most likely not a software problem. os_info: :NAME="Springdale Linux" :VERSION="7.6 (Verona)" :ID="rhel" :ID_LIKE="fedora" :VERSION_ID="7.6" :PRETTY_NAME="Springdale Linux 7.6 (Verona)" :ANSI_COLOR="0;32" :CPE_NAME="cpe:/o:springdale:linux:7.6:GA" :HOME_URL="http://springdale.princeton.edu/" :BUG_REPORT_URL="https://springdale.math.ias.edu/" : :REDHAT_BUGZILLA_PRODUCT="Springdale Linux 7" :REDHAT_BUGZILLA_PRODUCT_VERSION=7.6 :REDHAT_SUPPORT_PRODUCT="Springdale Linux" :REDHAT_SUPPORT_PRODUCT_VERSION=7.6 proc_modules: :coretemp 13444 0 - Live 0xffffffffc099c000 :ebtable_filter 12827 0 - Live 0xffffffffc08a7000 :ebtables 35009 1 ebtable_filter, Live 0xffffffffc09d0000 :ip6table_filter 12815 0 - Live 0xffffffffc0873000 :ip6_tables 26912 1 ip6table_filter, Live 0xffffffffc09b9000 :devlink 42368 0 - Live 0xffffffffc09a1000 :nfsv3 43720 2 - Live 0xffffffffc08bb000 :nfs_acl 12837 1 nfsv3, Live 0xffffffffc086e000 :rpcsec_gss_krb5 35549 0 - Live 0xffffffffc08b1000 :auth_rpcgss 59415 1 rpcsec_gss_krb5, Live 0xffffffffc0897000 :nfsv4 575138 0 - Live 0xffffffffc0abe000 :dns_resolver 13140 1 nfsv4, Live 0xffffffffc0869000 :nfs 261660 10 nfsv3,nfsv4, Live 0xffffffffc0a7d000 :lockd 93827 2 nfsv3,nfs, Live 0xffffffffc087f000 :grace 13515 1 lockd, Live 0xffffffffc0825000 :fscache 64984 2 nfsv4,nfs, Live 0xffffffffc0858000 :ipt_REJECT 12541 2 - Live 0xffffffffc07e4000 :nf_reject_ipv4 13373 1 ipt_REJECT, Live 0xffffffffc0820000 :nf_conntrack_ipv4 15053 3 - Live 0xffffffffc07d8000 :nf_defrag_ipv4 12729 1 nf_conntrack_ipv4, Live 0xffffffffc07d3000 :xt_conntrack 12760 3 - Live 0xffffffffc0788000 :nf_conntrack 133053 2 nf_conntrack_ipv4,xt_conntrack, Live 0xffffffffc0830000 :iptable_filter 12810 1 - Live 0xffffffffc0783000 :sunrpc 353310 35 nfsv3,nfs_acl,rpcsec_gss_krb5,auth_rpcgss,nfsv4,nfs,lockd, Live 0xffffffffc0a0d000 :dm_mirror 22289 0 - Live 0xffffffffc077c000 :dm_region_hash 20813 1 dm_mirror, Live 0xffffffffc0775000 :dm_log 18411 2 dm_mirror,dm_region_hash, Live 0xffffffffc076f000 :dm_mod 123936 2 dm_mirror,dm_log, Live 0xffffffffc07a8000 :sb_edac 32034 0 - Live 0xffffffffc09c7000 :intel_powerclamp 14419 0 - Live 0xffffffffc0a08000 :raid456 142416 1 - Live 0xffffffffc0978000 :intel_rapl 19542 0 - Live 0xffffffffc09ad000 :iosf_mbi 14990 1 intel_rapl, Live 0xffffffffc0765000 :async_raid6_recov 17288 1 raid456, Live 0xffffffffc074f000 :iTCO_wdt 13480 0 - Live 0xffffffffc074a000 :async_memcpy 12768 2 raid456,async_raid6_recov, Live 0xffffffffc0734000 :kvm_intel 183720 0 - Live 0xffffffffc1fd3000 :iTCO_vendor_support 13718 1 iTCO_wdt, Live 0xffffffffc072f000 :kvm 578558 1 kvm_intel, Live 0xffffffffc08e9000 :async_pq 13332 2 raid456,async_raid6_recov, Live 0xffffffffc072a000 :irqbypass 13503 1 kvm, Live 0xffffffffc0725000 :raid6_pq 102527 3 raid456,async_raid6_recov,async_pq, Live 0xffffffffc08ce000 :crc32_pclmul 13133 0 - Live 0xffffffffc0704000 :async_xor 13127 3 raid456,async_raid6_recov,async_pq, Live 0xffffffffc06ff000 :xor 21411 1 async_xor, Live 0xffffffffc0878000 :ghash_clmulni_intel 13273 0 - Live 0xffffffffc04f7000 :aesni_intel 189415 0 - Live 0xffffffffc07f0000 :async_tx 13509 5 raid456,async_raid6_recov,async_memcpy,async_pq,async_xor, Live 0xffffffffc082b000 :lrw 13286 1 aesni_intel, Live 0xffffffffc0853000 :gf128mul 15139 1 lrw, Live 0xffffffffc07df000 :glue_helper 13990 1 aesni_intel, Live 0xffffffffc07eb000 :ablk_helper 13597 1 aesni_intel, Live 0xffffffffc08ac000 :cryptd 20511 3 ghash_clmulni_intel,aesni_intel,ablk_helper, Live 0xffffffffc0743000 :ipmi_si 57587 1 - Live 0xffffffffc0715000 :joydev 17389 0 - Live 0xffffffffc05cc000 :ipmi_devintf 17603 2 - Live 0xffffffffc05b8000 :ipmi_msghandler 46607 2 ipmi_si,ipmi_devintf, Live 0xffffffffc0758000 :sg 40721 0 - Live 0xffffffffc070a000 :i2c_i801 22550 0 - Live 0xffffffffc05b1000 :pcspkr 12718 0 - Live 0xffffffffc04c1000 :lpc_ich 21086 0 - Live 0xffffffffc0494000 :shpchp 37047 0 - Live 0xffffffffc07c8000 :ioatdma 67809 64 - Live 0xffffffffc059f000 :mei_me 32848 0 - Live 0xffffffffc0739000 :mei 91099 1 mei_me, Live 0xffffffffc0790000 :wmi 19086 0 - Live 0xffffffffc08c8000 :binfmt_misc 17468 1 - Live 0xffffffffc04aa000 :ip_tables 27126 1 iptable_filter, Live 0xffffffffc048c000 :xfs 1003971 4 - Live 0xffffffffc0608000 :libcrc32c 12644 3 nf_conntrack,raid456,xfs, Live 0xffffffffc04a5000 :raid1 39929 3 - Live 0xffffffffc0468000 :sd_mod 46322 13 - Live 0xffffffffc04b4000 :crc_t10dif 12912 1 sd_mod, Live 0xffffffffc049c000 :crct10dif_generic 12647 0 - Live 0xffffffffc0487000 :mgag200 41138 1 - Live 0xffffffffc052e000 :drm_kms_helper 177166 1 mgag200, Live 0xffffffffc0501000 :syscopyarea 12529 1 drm_kms_helper, Live 0xffffffffc04ed000 :sysfillrect 12701 1 drm_kms_helper, Live 0xffffffffc04e8000 :sysimgblt 12640 1 drm_kms_helper, Live 0xffffffffc04e3000 :fb_sys_fops 12703 1 drm_kms_helper, Live 0xffffffffc03d7000 :ttm 99555 1 mgag200, Live 0xffffffffc04c9000 :igb 210385 0 - Live 0xffffffffc05d3000 :ahci 34056 0 - Live 0xffffffffc05c2000 :drm 397988 4 mgag200,drm_kms_helper,ttm, Live 0xffffffffc053c000 :crct10dif_pclmul 14307 1 - Live 0xffffffffc04fc000 :crct10dif_common 12595 3 crc_t10dif,crct10dif_generic,crct10dif_pclmul, Live 0xffffffffc04f2000 :crc32c_intel 22094 1 - Live 0xffffffffc0419000 :libahci 31992 1 ahci, Live 0xffffffffc0423000 :ptp 19231 1 igb, Live 0xffffffffc0479000 :pps_core 19057 1 ptp, Live 0xffffffffc0481000 :dca 15130 2 ioatdma,igb, Live 0xffffffffc0474000 :mpt3sas 222977 17 - Live 0xffffffffc0430000 :libata 243070 2 ahci,libahci, Live 0xffffffffc03dc000 :i2c_algo_bit 13413 2 mgag200,igb, Live 0xffffffffc03b7000 :i2c_core 63151 6 i2c_i801,mgag200,drm_kms_helper,igb,drm,i2c_algo_bit, Live 0xffffffffc03c6000 :raid_class 13554 1 mpt3sas, Live 0xffffffffc03be000 :scsi_transport_sas 41224 1 mpt3sas, Live 0xffffffffc03ab000 suspend_stats: :success: 0 :fail: 0 :failed_freeze: 0 :failed_prepare: 0 :failed_suspend: 0 :failed_suspend_late: 0 :failed_suspend_noirq: 0 :failed_resume: 0 :failed_resume_early: 0 :failed_resume_noirq: 0 :failures: : last_failed_dev: : : last_failed_errno: 0 : 0 : last_failed_step: : From auton.sysnotify at gmail.com Thu Nov 29 23:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Thu, 29 Nov 2018 23:30:05 -0500 Subject: A desktop is down Message-ID: <20181130043006.48B53412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 00:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 00:00:06 -0500 Subject: A desktop is down Message-ID: <20181130050006.D512B412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 00:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 00:30:06 -0500 Subject: A desktop is down Message-ID: <20181130053007.0EFED412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 01:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 01:00:06 -0500 Subject: A desktop is down Message-ID: <20181130060007.012E7412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 01:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 01:30:06 -0500 Subject: A desktop is down Message-ID: <20181130063006.8D799412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 02:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 02:00:06 -0500 Subject: A desktop is down Message-ID: <20181130070006.4B0EC412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 02:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 02:30:06 -0500 Subject: A desktop is down Message-ID: <20181130073007.30AE8412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 03:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 03:00:06 -0500 Subject: A desktop is down Message-ID: <20181130080006.ED7CE412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 03:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 03:30:06 -0500 Subject: A desktop is down Message-ID: <20181130083006.C26E3412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 04:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 04:00:06 -0500 Subject: A desktop is down Message-ID: <20181130090006.8183B412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 04:30:05 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 04:30:05 -0500 Subject: A desktop is down Message-ID: <20181130093005.E7771412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 05:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 05:00:06 -0500 Subject: A desktop is down Message-ID: <20181130100006.C355F412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 05:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 05:30:06 -0500 Subject: A desktop is down Message-ID: <20181130103006.5B67C412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 06:00:00 2018 From: auton.sysnotify at gmail.com (Cron Daemon) Date: Fri, 30 Nov 2018 06:00:00 -0500 Subject: Cron /usr/local/sbin/zxfer -dFkP -o compression=lzjb -T root@backup1 -R zroot/iocell storage Message-ID: <5c011830.201b1.101e2bd3@bhyve.int.autonlab.org> ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known ssh: Could not resolve hostname backup1: hostname nor servname provided, or not known Failed to retrieve datasets from the destination From auton.sysnotify at gmail.com Fri Nov 30 06:00:05 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 06:00:05 -0500 Subject: A desktop is down Message-ID: <20181130110005.E09D8412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 06:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 06:30:06 -0500 Subject: A desktop is down Message-ID: <20181130113006.CA227412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 07:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 07:00:06 -0500 Subject: A desktop is down Message-ID: <20181130120006.B3AD4412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 07:30:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 07:30:06 -0500 Subject: A desktop is down Message-ID: <20181130123006.CAF83412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu From auton.sysnotify at gmail.com Fri Nov 30 08:00:06 2018 From: auton.sysnotify at gmail.com (root) Date: Fri, 30 Nov 2018 08:00:06 -0500 Subject: A desktop is down Message-ID: <20181130130006.4AC08412B48F@lake.auton.cs.cmu.edu> lori.auton.cs.cmu.edu